NETOBSERV-2225 - Deploy static plugin at operator startup #1345

jpinsonneau · 2025-04-02T09:38:40Z

Description

Create the console plugin when FlowCollector doesn't exists to expose the new forms.

~~Suggested alternatives: #1346 & #1374~~

See netobserv/network-observability-console-plugin#763 for the forms implementations

Dependencies

NETOBSERV-1942 config / status views + NETOBSERV-1945 resource calculator network-observability-console-plugin#763

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

jpinsonneau · 2025-04-02T10:05:44Z

controllers/flowcollector_controller.go

+	// force reconcile at startup
+	go r.InitReconcile(ctx)
+
 	return nil
 }

+func (r *FlowCollectorReconciler) InitReconcile(ctx context.Context) error {
+	log := log.FromContext(ctx)
+	log.Info("Initializing resources...")
+
+	var err error
+	for attempt := range initReconcileAttempts {
+		// delay the reconcile calls to let some time to the cache to load
+		time.Sleep(5 * time.Second)
+		_, err = r.Reconcile(ctx, reconcile.Request{})
+		if err != nil {
+			log.Error(err, "Error while doing initial reconcile", "attempt", attempt)
+		} else {
+			break
+		}
+	}
+	return err
+}


☝️ I wonder if there is an out of box mechanism to trigger the loop after the cache loaded. That's why I'm using a sleep here and this will may work in all situations.

https://redhat-internal.slack.com/archives/C02939DP5L5/p1743518264445429

If I remember correctly, when a reconcile loop is failing, you can also return a time value to reschedule the reconciliation.

Yeah I gave a try with that without success.

I'm refactoring the code again to move the static content to another controller wich will be cleaner I guess. I will give another try with the reschedule time on the new controller 👍

jpinsonneau · 2025-04-17T08:24:43Z

controllers/static/static_controller.go

+	// force reconcile at startup
+	go r.InitReconcile(ctx)
+
+	return ctrl.NewControllerManagedBy(mgr).
+		For(&flowslatest.FlowCollector{}, reconcilers.IgnoreStatusChange).
+		Named("staticPlugin").
+		Complete(&r)
+}
+
+func (r *Reconciler) InitReconcile(ctx context.Context) {
+	log := log.FromContext(ctx)
+	log.Info("Initializing resources...")
+
+	for attempt := range initReconcileAttempts {
+		// delay the reconcile calls to let some time to the cache to load
+		time.Sleep(5 * time.Second)
+		_, err := r.Reconcile(ctx, ctrl.Request{})
+		if err != nil {
+			log.Error(err, "Error while doing initial reconcile", "attempt", attempt)
+		} else {
+			return
+		}
+	}
+}


@OlivierCazade I modified the PR to be in a dedicated controller.

As you can see, I force the Reconcile call during the Start function so OLM can't interpret the result containing Requeue / RequeueAfter here.

Since we don't create a dedicated CR for static plugin, I don't think we can rely on these here.

WDYT ?

jpinsonneau · 2025-07-15T08:41:28Z

Rebased without changes

memodi · 2025-07-15T20:10:50Z

/ok-to-test

github-actions · 2025-07-15T20:15:08Z

New images:

quay.io/netobserv/network-observability-operator:cdd8f5a
quay.io/netobserv/network-observability-operator-bundle:v0.0.0-sha-cdd8f5a
quay.io/netobserv/network-observability-operator-catalog:v0.0.0-sha-cdd8f5a

They will expire after two weeks.

To deploy this build:

# Direct deployment, from operator repo
IMAGE=quay.io/netobserv/network-observability-operator:cdd8f5a make deploy

# Or using operator-sdk
operator-sdk run bundle quay.io/netobserv/network-observability-operator-bundle:v0.0.0-sha-cdd8f5a

Or as a Catalog Source:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: netobserv-dev
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/netobserv/network-observability-operator-catalog:v0.0.0-sha-cdd8f5a
  displayName: NetObserv development catalog
  publisher: Me
  updateStrategy:
    registryPoll:
      interval: 1m

memodi · 2025-07-15T21:00:58Z

@jpinsonneau Here's quick feedback from my initial review of the form view:

- ebpf flow filtering
	- what does option 1 and 2 mean for port fields? [1]
- Deployment model
- Mode:
	- add a note which Loki mode is recommended for production use cases. 
- Prometheus Mode:
	- add a node what "Auto" mode mean? Or hide it if its not relevant 
- pipeline stage:
	- flow filtering.
	- some advanced configuration like "secondaryNetworks" could be exposed.

[1]

at the end it generated empty yaml [2] , I was trying with 4.20 OCP version

I'll add more feedback as I test more.

memodi · 2025-07-16T16:38:45Z

bundle/manifests/netobserv-operator.clusterserviceversion.yaml

+                - name: NAMESPACE
+                  valueFrom:
+                    fieldRef:
+                      fieldPath: metadata.namespace


@jpinsonneau - can we append this env var instead of adding it in middle so that index # for the existing vars doesn't change? We have several scripts and CI steps where we patch csv which happens based on index of env var.

Sure, I cant put it in the end of the list if it help 😸

memodi · 2025-07-17T02:36:19Z

Adding more feedback:

A way to skip through all the config options would be nice for users who are looking to just configure with default config.
Is there a way to have a hyperlink of a diagram presented on [k8s/cluster/flows.netobserv.io~~v1beta2~~FlowCollector/status] of each component for the "status" field shown on [flows.netobserv.io~~v1beta2~~FlowCollector] page
the view port of the diagram is very small on my 16" laptop screen

When I used kafka mode, the FLP component never went green even though it was working fine.
nit: the lines between FLP --> Loki/Prom. keeps flickering.

Screen.Recording.2025-07-16.at.10.28.05.PM.mov

jpinsonneau · 2025-07-17T10:25:53Z

@jpinsonneau Here's quick feedback from my initial review of the form view:

- ebpf flow filtering
	- what does option 1 and 2 mean for port fields? [1]
- Deployment model
- Mode:
	- add a note which Loki mode is recommended for production use cases. 
- Prometheus Mode:
	- add a node what "Auto" mode mean? Or hide it if its not relevant 
- pipeline stage:
	- flow filtering.
	- some advanced configuration like "secondaryNetworks" could be exposed.

Sure I can add some notes and recommendations on top of the current descriptions. These are coming from the API so maybe we can also improve it.

About the options 1 / 2 and the advanced config, we need to decide what to expose and what to skip as these needs extra tweaking to make it work.
For example, options 1 & 2 are about using a single port or an array or range:
https://github.com/netobserv/network-observability-operator/blob/main/internal/controller/ebpf/agent_controller.go#L560-L572

If we decide to not expose these, I will simply hide these fields.

jpinsonneau · 2025-07-17T10:36:13Z

A way to skip through all the config options would be nice for users who are looking to just configure with default config.

You can directly click on the left menu to navigate to the last step. Isn't it enough ?

Is there a way to have a hyperlink of a diagram presented on [k8s/cluster/flows.netobserv.io~~v1beta2~~FlowCollector/status] of each component for the "status" field shown on [flows.netobserv.io~~v1beta2~~FlowCollector] page

Sure, good idea !

the view port of the diagram is very small on my 16" laptop screen

Indeed. Would you prefer to have a scrollbar on the status list or a scrollbar on the entire page ?

When I used kafka mode, the FLP component never went green even though it was working fine.

Indeed I should rely on the WaitingFLPTransformer for kafka:
https://github.com/netobserv/network-observability-console-plugin/pull/763/files
I'll do the change

nit: the lines between FLP --> Loki/Prom. keeps flickering.

I can see if I can skip some refreshs here to avoid that. Thanks for your feedback !

memodi · 2025-07-17T14:45:29Z

You can directly click on the left menu to navigate to the last step. Isn't it enough ?

ah, it wasn't clear me, that I could jump that way.

Would you prefer to have a scrollbar on the status list or a scrollbar on the entire page ?

I think scrollbar on status list make sense to me.

About the options 1 / 2 and the advanced config, we need to decide what to expose and what to skip as these needs extra tweaking to make it work.
For example, options 1 & 2 are about using a single port or an array or range:
https://github.com/netobserv/network-observability-operator/blob/main/internal/controller/ebpf/agent_controller.go#L560-L572
If we decide to not expose these, I will simply hide these fields.

I think these are much advanced config which can be handled directly in yaml.

of all the advancedConfig, IMHO secondaryNetworks may probably be more utilized than others, may be it make a case to move it out of advanced config in API for it.

memodi · 2025-07-17T18:46:38Z

/label needs-changes

openshift-ci · 2025-07-17T18:46:41Z

@memodi: The label(s) /label needs-changes cannot be applied. These labels are supported: acknowledge-critical-fixes-only, platform/aws, platform/azure, platform/baremetal, platform/google, platform/libvirt, platform/openstack, ga, tide/merge-method-merge, tide/merge-method-rebase, tide/merge-method-squash, px-approved, docs-approved, qe-approved, ux-approved, no-qe, downstream-change-needed, rebase/manual, cluster-config-api-changed, run-integration-tests, approved, backport-risk-assessed, bugzilla/valid-bug, cherry-pick-approved, jira/valid-bug, stability-fix-approved, staff-eng-approved. Is this label configured under labels -> additional_labels or labels -> restricted_labels in plugin.yaml?

In response to this:

/label needs-changes

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

memodi · 2025-07-17T18:46:56Z

/needs-changes

jpinsonneau · 2025-07-18T12:06:46Z

@memodi coming back on the notes, that's what we have today:

I would suggest to update the CRD first and then in the forms if we are not satisfied with those texts

memodi · 2025-07-18T13:50:42Z

I would suggest to update the CRD first and then in the forms if we are not satisfied with those texts

sure, np, though it would be nice to have a way to override API docs text

Amoghrd · 2025-07-22T15:18:36Z

/ok-to-test

github-actions · 2025-07-22T15:23:17Z

New images:

quay.io/netobserv/network-observability-operator:02a5af9
quay.io/netobserv/network-observability-operator-bundle:v0.0.0-sha-02a5af9
quay.io/netobserv/network-observability-operator-catalog:v0.0.0-sha-02a5af9

They will expire after two weeks.

To deploy this build:

# Direct deployment, from operator repo
IMAGE=quay.io/netobserv/network-observability-operator:02a5af9 make deploy

# Or using operator-sdk
operator-sdk run bundle quay.io/netobserv/network-observability-operator-bundle:v0.0.0-sha-02a5af9

Or as a Catalog Source:

apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: netobserv-dev
  namespace: openshift-marketplace
spec:
  sourceType: grpc
  image: quay.io/netobserv/network-observability-operator-catalog:v0.0.0-sha-02a5af9
  displayName: NetObserv development catalog
  publisher: Me
  updateStrategy:
    registryPoll:
      interval: 1m

openshift-ci · 2025-07-22T17:54:31Z

@jpinsonneau: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-operator	`9bb32d1`	link	false	`/test e2e-operator`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

jotak

/lgtm

jotak · 2025-07-24T14:41:05Z

another round of testing will be done post-merge; merging now!

jpinsonneau force-pushed the 1942 branch from 0b47945 to efab0a0 Compare April 2, 2025 10:04

jpinsonneau commented Apr 2, 2025

View reviewed changes

jotak added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 2, 2025

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 2, 2025

jpinsonneau added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 2, 2025

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 2, 2025

jotak mentioned this pull request Apr 7, 2025

Create static console plugin in OLM bundle #1346

Closed

jpinsonneau force-pushed the 1942 branch from 5fb53db to c5da751 Compare April 14, 2025 09:04

netobserv deleted a comment from github-actions bot Apr 14, 2025

jpinsonneau added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 14, 2025

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 14, 2025

jpinsonneau force-pushed the 1942 branch 2 times, most recently from c4c57e1 to dbc4cbf Compare April 15, 2025 10:40

netobserv deleted a comment from github-actions bot Apr 15, 2025

jpinsonneau added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 15, 2025

jpinsonneau force-pushed the 1942 branch from dbc4cbf to 82efd81 Compare April 15, 2025 10:59

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 15, 2025

netobserv deleted a comment from github-actions bot Apr 15, 2025

openshift-merge-robot added the needs-rebase label Apr 15, 2025

jpinsonneau added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 15, 2025

jpinsonneau force-pushed the 1942 branch from 75f2aac to 82efd81 Compare April 15, 2025 12:39

openshift-merge-robot removed the needs-rebase label Apr 15, 2025

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 15, 2025

netobserv deleted a comment from github-actions bot Apr 15, 2025

jpinsonneau added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 15, 2025

jpinsonneau force-pushed the 1942 branch from 82efd81 to 0a42edc Compare April 17, 2025 08:21

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Apr 17, 2025

jpinsonneau commented Apr 17, 2025

View reviewed changes

netobserv deleted a comment from github-actions bot Apr 17, 2025

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Jul 15, 2025

memodi reviewed Jul 16, 2025

View reviewed changes

memodi added the needs-changes To be added to denote PR needs changes or some questions/comments to be addressed label Jul 17, 2025

memodi mentioned this pull request Jul 17, 2025

NETOBSERV-1942 config / status views + NETOBSERV-1945 resource calculator netobserv/network-observability-console-plugin#763

Merged

10 tasks

namespace env at the end of the list for CI

9bb32d1

github-actions bot removed the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Jul 18, 2025

openshift-ci bot added the ok-to-test To set manually when a PR is safe to test. Triggers image build on PR. label Jul 22, 2025

jpinsonneau removed the needs-changes To be added to denote PR needs changes or some questions/comments to be addressed label Jul 23, 2025

jotak approved these changes Jul 23, 2025

View reviewed changes

openshift-ci bot added the lgtm label Jul 23, 2025

jotak merged commit 5c7af20 into main Jul 24, 2025
13 of 15 checks passed

jpinsonneau mentioned this pull request Jul 25, 2025

NETOBSERV-2225 - Deploy static plugin at operator startup - fix #1789

Merged

10 tasks

jpinsonneau deleted the 1942 branch October 1, 2025 11:05

NETOBSERV-2225 - Deploy static plugin at operator startup #1345

NETOBSERV-2225 - Deploy static plugin at operator startup #1345

Uh oh!

Conversation

jpinsonneau commented Apr 2, 2025 • edited by jotak Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Dependencies

Checklist

Uh oh!

jpinsonneau Apr 2, 2025

Choose a reason for hiding this comment

Uh oh!

OlivierCazade Apr 16, 2025

Choose a reason for hiding this comment

Uh oh!

jpinsonneau Apr 16, 2025

Choose a reason for hiding this comment

Uh oh!

jpinsonneau Apr 17, 2025

Choose a reason for hiding this comment

Uh oh!

jpinsonneau commented Jul 15, 2025

Uh oh!

memodi commented Jul 15, 2025

Uh oh!

github-actions bot commented Jul 15, 2025

Uh oh!

memodi commented Jul 15, 2025

Uh oh!

memodi Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jpinsonneau Jul 17, 2025

Choose a reason for hiding this comment

Uh oh!

jpinsonneau Jul 18, 2025

Choose a reason for hiding this comment

Uh oh!

memodi commented Jul 17, 2025

Uh oh!

jpinsonneau commented Jul 17, 2025

Uh oh!

jpinsonneau commented Jul 17, 2025

Uh oh!

memodi commented Jul 17, 2025

Uh oh!

memodi commented Jul 17, 2025

Uh oh!

openshift-ci bot commented Jul 17, 2025

Uh oh!

memodi commented Jul 17, 2025

Uh oh!

jpinsonneau commented Jul 18, 2025

Uh oh!

memodi commented Jul 18, 2025

Uh oh!

Amoghrd commented Jul 22, 2025

Uh oh!

github-actions bot commented Jul 22, 2025

Uh oh!

openshift-ci bot commented Jul 22, 2025

Uh oh!

jotak left a comment

Choose a reason for hiding this comment

Uh oh!

jotak commented Jul 24, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

jpinsonneau commented Apr 2, 2025 •

edited by jotak

Loading

memodi Jul 16, 2025 •

edited

Loading