Helm: pipeline integration

[jotak]

Follow-up on adding a helm chart (#1000)

Currently, the helm chart is only static files. We should make them more integrated:

• With make targets that update the container image tags
• Check how to integrate that into ArtifactHub
• If possible, automate publishing on ArtifactHub via our Github action "release" workflow
•  Declare cert-manager as a dependency: https://helm.sh/docs/chart_best_practices/dependencies/

[jotak]

doing some preliminary work here: #1113

[Helion55]

Hello @jotak Is there anything you want me to work on regarding this issue?

[jotak]

Hi @Helion55 ,
thanks for proposing your help again!
We're currently looking into publishing the 1.8.0 community version, I'm going to push the related images (the ones mentioned here)
In #1113 I'm adding some automation for updating the helm charts on new releases, mostly here : scripting some of the manual changes that we did previously in your PR

Now, the part that I'm less familiar with, is ArtifactHub. If I'm correct, this is the go-to solution for anyone who want to publish helm charts, right? Maybe you're more familiar with it than me. Basically, I'm reading this:

• https://artifacthub.io/docs/topics/repositories/helm-charts/
• https://helm.sh/docs/topics/chart_repository/

So either we need to have a server that exposes the charts + an index.yaml, or we can do it via an OCI registry. We already serve our images via quay.io which I guess can be used as the OCI registry, but I don't know at this point what's better for us between the server or the OCI registry - if you happen to have some experience already in that field, I'm happy to get your advice! Else we can figure out together.

We also have github pages set up ( https://github.com/netobserv/netobserv.github.io / https://netobserv.io/ ) so that can probably be used for the non-OCI alternative.

[Helion55]

Thanks @jotak .Got the idea, starting to work on the ArtifactHub publishing template. I think It required a much more dynamic chart than it is right now. Share your thoughts on that.

So either we need to have a server that exposes the charts + an index.yaml, or we can do it via an OCI registry. We already serve our images via quay.io which I guess can be used as the OCI registry, but I don't know at this point what's better for us between the server or the OCI registry - if you happen to have some experience already in that field, I'm happy to get your advice! Else we can figure out together.

As I am also not too professional on this thing, I am also starting to explore a bit about it.

[Helion55]

Now, the part that I'm less familiar with, is ArtifactHub. If I'm correct, this is the go-to solution for anyone who want to publish helm charts, right? Maybe you're more familiar with it than me. Basically, I'm reading this:

• https://artifacthub.io/docs/topics/repositories/helm-charts/
• https://helm.sh/docs/topics/chart_repository/

As far as I know, Helm was previously maintaining a project called stable. It was the default, tested, stable charts for applications, but due to maintenance overhead and cost issues, this was deprecated, and ArtifactHub has now become the default solution for organizations to publish their own charts in their own maintenance.
You can also check them out:

https://helm.sh/blog/charts-repo-deprecation/
https://github.com/helm/charts
https://charts.helm.sh/stable/

[jotak]

The PR that adds helm template generation & github action checks is merged
Next is ArtifactHub :-)

[jotak]

Also I added the standalone console deployment as part of the helm configs, and updated the doc here: https://github.com/netobserv/network-observability-operator/tree/main?tab=readme-ov-file#install-with-helm

[jotak]

Here we are: https://artifacthub.io/packages/helm/netobserv/netobserv-operator

[jotak]

I'll add more automation later. Currently the process involves:

• Running make helm-update && helm package helm/ from the operator repo
• Copying the archive to the site repo
• Running helm repo index helm/ --url https://netobserv.io/static/helm/ from the site repo
• Revert the "name" field in the index.yaml
• Commit & push to site

Also, maybe more next steps (but I'm not sure):

• Fill out the chart dependencies? (https://helm.sh/docs/topics/charts/#chart-dependencies): it depends how dependencies are managed with helm ... like, does it makes packet XYZ mandatory? We may not want that, for instance, there's several ways to have prometheus installed, or to have some certificate issuers.
• Fill out the CRD sections? (https://helm.sh/docs/topics/charts/#custom-resource-definitions-crds): the docs says CRDs has to be 100% yaml (no injected vars), which won't work if we need Namespace injection for webhooks.
• Making it Official: https://artifacthub.io/docs/topics/repositories/#official-status

[Helion55]

Hello @jotak , I regret my absence to participate in those contributions; now I have started working on these two things.

• Fill out the chart dependencies? (https://helm.sh/docs/topics/charts/#chart-dependencies): it depends how dependencies are managed with helm ... like, does it makes packet XYZ mandatory? We may not want that, for instance, there's several ways to have prometheus installed, or to have some certificate issuers.

It can be configured in the Chart.yaml file to enable or disable the installation of the dependencies, and also we can toggle the value during installation. I will add that. And if I am right, we can give Prometheus, Loki, and cert-manager as dependencies, and then we will add the option to install or not.

• Fill out the CRD sections? (https://helm.sh/docs/topics/charts/#custom-resource-definitions-crds): the docs says CRDs has to be 100% yaml (no injected vars), which won't work if we need Namespace injection for webhooks.

CRDs are not namespaced; by mistake, the namespace attribute has been given, will rectify that. Correct me if I am getting this wrong related to webhooks; kindly guide me on that.

[jotak]

Hi @Helion55
No worries & thanks for your contributions already!

It can be configured in the Chart.yaml file to enable or disable the installation of the dependencies, and also we can toggle the value during installation. I will add that. And if I am right, we can give Prometheus, Loki, and cert-manager as dependencies, and then we will add the option to install or not.

Sounds good, thanks!

CRDs are not namespaced; by mistake, the namespace attribute has been given, will rectify that. Correct me if I am getting this wrong related to webhooks; kindly guide me on that.

The CRD itself is not namespaced, but the webhook definition, that is part of the CRD, depends on the namespace where the operator is installed, see here: https://github.com/netobserv/network-observability-operator/blob/main/helm/templates/flows.netobserv.io_flowcollectors.yaml#L11-L18. That's because it references the Service that is used to reach the webhook. I don't know if there would be other ways of doing it, but given how it is today, I don't think we can have a pure-static-yaml CRD definition.

[Helion55]

The CRD itself is not namespaced, but the webhook definition, that is part of the CRD, depends on the namespace where the operator is installed, see here: https://github.com/netobserv/network-observability-operator/blob/main/helm/templates/flows.netobserv.io_flowcollectors.yaml#L11-L18. That's because it references the Service that is used to reach the webhook. I don't know if there would be other ways of doing it, but given how it is today, I don't think we can have a pure-static-yaml CRD definition.

Got it. So what about deploying this whole netobsev helm deployment in a single namespace only. Like the ArgoCD, cert-manager, etc?