Add various Helm charts POCs #3154

kunal-511 · 2025-06-03T12:00:32Z

Pull Request Template for Kubeflow Manifests

✏️ Summary of Changes

Introduces Helm chart structure for Kubeflow components starting with spark-operator, including umbrella chart pattern and component-specific charts. Implements comprehensive verification system to ensure Helm produces functionally equivalent manifests to existing Kustomize approach with CI/CD validation.
Created experimental/helm/kubeflow/ umbrella chart with Chart.yaml, values.yaml
Created experimental/helm/charts/spark-operator/ component chart referencing upstream v2.2.0

🐛 Related Issues

Deploying Kubeflow with helm

✅ Contributor Checklist

I have tested these changes with kustomize. See Installation Prerequisites.
All commits are signed-off to satisfy the DCO check.
I have considered adding my company to the adopters page to support Kubeflow and help the community, since I expect help from the community for my issue (see 1. and 2.).

You can join the CNCF Slack and access our meetings at the Kubeflow Community website. Our channel on the CNCF Slack is here #kubeflow-platform.

google-oss-prow · 2025-06-03T12:00:40Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign kimwnasptd for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

andreyvelich · 2025-06-03T12:11:06Z

experimental/helm/kubeflow/values-example.yaml

+# Example(Template) custom values for Kubeflow deployment
+
+# Deploy spark-operator with custom configuration
+spark-operator:


Thank you for doing this @kunal-511.
I thought, we agreed that we are not going to have Helm Charts under kubeflow/manifests repository?
kubeflow/community#832 (comment)

Which means Helm Charts should be stored in the projects repos:

https://github.com/kubeflow/spark-operator/tree/master/charts/spark-operator-chart

https://github.com/kubeflow/trainer/tree/master/charts/kubeflow-trainer

cc @kubeflow/kubeflow-steering-committee @thesuperzapper @kimwnasptd

Agreed upon was that the individual per project helm charts (KFP, Katib, Dashboard, ...) have the single source of truth long-term in the individual project repositories. But first we need to develop and test them. We need to see how they work together and use the same CI/CD. then remove them step by step again from /experimental. That is how I can mentor this GSOC project. Otherwise i will just not have the time for it if there is too much coordination overhead.

But as Andrey said @kunal-511, since there is already a Spark and trainer helm chart, you need to copy it from upstream automatically Maybe write a general script as you did for the kustomize manifests. The other ones you would have to do from scratch and upstream later at the end of the GSOC project. Long term we will synchronize all the individual helm charts as we now synchronize the kustomize manifests. Therefore it also makes sense to have them in an upstream folder as we do for kustomize.

So i need to just create the 2 scripts that just fetch the upstream helm charts from there respective github ?? And add them under the experimental/helm/spark-operator and experimental/helm/trainer

For the existing helm charts yes. But most other components do not have helm charts and there you have to build them from scratch. In the best case we can make them somewhat standardized. Then test whether kustomize and helm render the same output.

Why do we need to duplicate Helm Charts from the individual repos in the kubeflow/manifests ? It would be hard to maintain 2 different Helm Charts and keep them in sync for every Kubeflow Project.

If you want to test installation for them, you can just define the script in kubeflow/manifests that deploys Helm Charts one after another from the upstream repositories.

As we discussed with @kimwnasptd and @thesuperzapper, kubeflow/manifests should not have Helm Charts manifests, since it focuses on Kustomize deployment.

@kunal-511 The Kubeflow community has not yet agreed for all-in-one Helm chart. That’s why we should begin by working on individual Helm charts for each Kubeflow tool:

Kubeflow Pipelines Kubeflow Trainer Kubeflow Katib Kubeflow Model Registry Kubeflow Notebooks Kubeflow Spark Operator

/hold

Well yes the individual ones come first. They must be seperately deployable but they should be later combinable by downstream users.

@juliusvonkohout Please can we keep this comment unresolved, since we still haven't agreed to have global Helm Chart for the entire Kubeflow Platform (e.g. Kubeflow Manifests) ?
cc @kimwnasptd @thesuperzapper

@kunal-511 please if there is still one left, remove the combining chart for political reasons for now. This PR is open for way too long and we need to get it merged tomorrow.

Okay I will just include the scripts for now and test them in the gha itself and not combining them.

kunal-511 · 2025-06-04T15:07:43Z

Please can you check is this the right way now @juliusvonkohout

kunal-511 · 2025-06-09T18:14:19Z

@juliusvonkohout do i make the sync helm scripts to be same as the kustomize one which creates a seperate pr for that or include them directly here ?

juliusvonkohout · 2025-06-13T10:22:28Z

CC @kromanow94 @chasecadet @varodrig

juliusvonkohout · 2025-06-23T14:42:19Z

.github/workflows/helm_charts_test.yaml

+
+    - name: Install Helm
+      run: |
+        chmod +x tests/helm_install.sh


can you fix this directly on the file system ?

juliusvonkohout · 2025-06-23T14:42:22Z

.github/workflows/helm_charts_test.yaml

+
+    - name: Run Helm Chart Tests
+      run: |
+        chmod +x experimental/helm/scripts/test-helm-charts.sh


can you fix this directly on the file system ?

juliusvonkohout · 2025-06-23T14:44:18Z

experimental/helm/kubeflow/Chart.yaml

+description: All-in-One Helm chart for deploying Kubeflow Platform
+type: application
+version: 0.1.0
+appVersion: "1.10.0"


master instead of 1.10.0 i assume

Signed-off-by: kunal-511 <[email protected]>

kunal-511 · 2025-06-23T19:12:29Z

Rebase to master

Signed-off-by: kunal-511 <[email protected]>

vikas-saxena02 · 2025-06-25T04:54:57Z

tests/helm_install.sh

+set -e
+
+echo "Installing Helm..."
+curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3


the https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 script will force an update of helm which may or may not be something that the user wants. IMHO its better to have own script which checks if helm is already installed or not and then prompt user to continue if they need. Also, the above script will always install updated version of helm which,in future, may not be compatible with our helm chart

Okay i will look for alternative

But please no interactive stuff. Just install a fixed helm version.

Signed-off-by: kunal-511 <[email protected]>

juliusvonkohout · 2025-07-08T14:29:01Z

@kunal-511 @andreyvelich what is blocking the PR now ?

juliusvonkohout · 2025-07-08T14:45:21Z

CC @chasecadet @varodrig

github-actions · 2025-09-07T00:23:57Z

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

juliusvonkohout · 2025-09-26T13:07:58Z

@kunal-511 please rebase and get this read to merge.

google-oss-prow bot requested review from juliusvonkohout and kimwnasptd June 3, 2025 12:00

google-oss-prow bot added the size/L label Jun 3, 2025

andreyvelich reviewed Jun 3, 2025

View reviewed changes

google-oss-prow bot added size/XL and removed size/L labels Jun 4, 2025

google-oss-prow bot added size/XXL and removed size/XL labels Jun 11, 2025

juliusvonkohout requested review from chasecadet and varodrig June 13, 2025 10:22

google-oss-prow bot added the do-not-merge/hold label Jun 14, 2025

juliusvonkohout changed the title ~~Add Helm chart support for Kubeflow spark-operator with automated verification~~ Add various Helm charts POCs Jun 23, 2025

juliusvonkohout reviewed Jun 23, 2025

View reviewed changes

kunal-511 added 10 commits June 24, 2025 00:40

Helm chart support for Kubeflow spark-operator

880c28c

Signed-off-by: kunal-511 <[email protected]>

Updated permission and formatting

1dc5372

Signed-off-by: kunal-511 <[email protected]>

Added the helm sync script for spark-operator

9fdb2d2

Signed-off-by: kunal-511 <[email protected]>

Added the spark sync script and updated the strucute according to AIO

9bdb472

Signed-off-by: kunal-511 <[email protected]>

Updated the name

66f88d9

Signed-off-by: kunal-511 <[email protected]>

Updated the comparision

3fb12ec

Signed-off-by: kunal-511 <[email protected]>

helm chart test updated

ecc1ed9

Signed-off-by: kunal-511 <[email protected]>

helm charts script for cert-manager

00c6e96

Signed-off-by: kunal-511 <[email protected]>

Handling the crds correctly in helm folder strucuture

5f0be99

Signed-off-by: kunal-511 <[email protected]>

Added central dashboard helm templates

89a77b0

Signed-off-by: kunal-511 <[email protected]>

kunal-511 force-pushed the feat/helm-spark-operator-with-verification branch from 531a341 to 89a77b0 Compare June 23, 2025 19:11

kunal-511 added 3 commits June 24, 2025 10:15

Updated the path to applications

3de1913

Signed-off-by: kunal-511 <[email protected]>

Updated to applications

0683d18

Signed-off-by: kunal-511 <[email protected]>

Updated script permissions

eb966fa

Signed-off-by: kunal-511 <[email protected]>

vikas-saxena02 reviewed Jun 25, 2025

View reviewed changes

kunal-511 added 6 commits June 25, 2025 18:36

Helm install a particular version

a00a521

Signed-off-by: kunal-511 <[email protected]>

Updated comparision to remove other crds when comparing

585a558

Signed-off-by: kunal-511 <[email protected]>

Spark-operator and cert-manager files from sync scripts

54bde97

Signed-off-by: kunal-511 <[email protected]>

removed the helm templates generated from the sync script

69e6b90

Signed-off-by: kunal-511 <[email protected]>

removed the helm templates generated from the sync script

47eb3d0

Signed-off-by: kunal-511 <[email protected]>

Removed the combination of the charts

fd26144

Signed-off-by: kunal-511 <[email protected]>

google-oss-prow bot added size/XL and removed size/XXL labels Jun 30, 2025

Updated the workflow

a415803

Signed-off-by: kunal-511 <[email protected]>

github-actions bot added the lifecycle/stale label Sep 7, 2025

github-actions bot removed the lifecycle/stale label Sep 27, 2025

kunal-511 closed this Sep 30, 2025

Add various Helm charts POCs #3154

Add various Helm charts POCs #3154

Conversation

kunal-511 commented Jun 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Template for Kubeflow Manifests

✏️ Summary of Changes

🐛 Related Issues

✅ Contributor Checklist

Uh oh!

google-oss-prow bot commented Jun 3, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

juliusvonkohout Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kunal-511 Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

juliusvonkohout Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

juliusvonkohout Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kunal-511 commented Jun 4, 2025

Uh oh!

kunal-511 commented Jun 9, 2025

Uh oh!

juliusvonkohout commented Jun 13, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kunal-511 commented Jun 23, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

juliusvonkohout commented Jul 8, 2025

Uh oh!

juliusvonkohout commented Jul 8, 2025

Uh oh!

github-actions bot commented Sep 7, 2025

Uh oh!

juliusvonkohout commented Sep 26, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

kunal-511 commented Jun 3, 2025 •

edited

Loading

juliusvonkohout Jun 4, 2025 •

edited

Loading

kunal-511 Jun 4, 2025 •

edited

Loading

juliusvonkohout Jun 4, 2025 •

edited

Loading

juliusvonkohout Jun 30, 2025 •

edited

Loading