feat: Add support for deploying K8s Agent as Helm Addon during cluster deployment #1374
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
**What problem does this PR solve?**: **Which issue(s) this PR fixes**: Fixes # **How Has This Been Tested?**: <!-- Please describe the tests that you ran to verify your changes. Provide output from the tests and any manual steps needed to replicate the tests. --> **Special notes for your reviewer**: <!-- Use this to provide any additional information to the reviewers. This may include: - Best way to review the PR. - Where the author wants the most review attention on. - etc. --> --------- Co-authored-by: Manoj Surudwad <[email protected]> Co-authored-by: Dimitri Koshkin <[email protected]>
dkoshkin
approved these changes
Nov 2, 2025
Contributor
dkoshkin
left a comment
dkoshkin
left a comment
There was a problem hiding this comment.
Can we just push this commit to the other branch and keep the original PR?
Contributor
Author
|
Merged #1345. so closing this PR |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Note
All commit history, discussions, and context for this change are available in PR #1345.
Due to GitHub’s signature verification requirements, I have recreated this PR with a single verified commit.
The code changes remain identical to PR #1345
What problem does this PR solve?:
This PR introduces support for deploying the Nutanix K8s Agent as a Helm-based addon as part of the cluster deployment workflow in the CAREN repository.
Key changes:
Added support for HelmAddon strategy in the K8s Registration Agent handler.
Updated handler logic to fetch and apply Helm chart configuration dynamically using the provided credentials.
Created unit tests to validate HelmAddon flow and error scenarios.
Ensured the agent’s credentials are handled securely through Kubernetes Secrets instead of directly passing via values.yaml.
This enhancement enables seamless installation of the K8s Agent via HelmReleaseProxy (HRP) and HelmChartProxy (HCP) without manual post-deployment intervention.
Motivation
Previously, the K8s Agent installation was not integrated into the cluster creation process and required manual setup.
By integrating it as an addon:
The K8s Agent will be automatically deployed during cluster provisioning.
Sensitive credentials are now secured using Kubernetes Secrets.
This aligns the agent deployment with other addon-based lifecycle management mechanisms.
Which issue(s) this PR fixes:
Fixes #
How Has This Been Tested?:
Verified K8s Agent deployment using the HelmAddon strategy.
Confirmed that:
Pre-install hooks execute as expected.
The secret is created and retained after installation.
Post-upgrade hooks are triggered correctly.
Added unit tests under k8sregistrationagent/handler_test.go
K8s agent documentation
Special notes for your reviewer:
Cluster class used for testing with konnector Agent:
ClusterConfigWIthKonnectorAgent.yaml
Cluster creation - with konnector agent secret and addon:
(devbox) vijayaraghavan.r@CQGK42CXL4 cluster-api-runtime-extensions-nutanix % kubectl apply -f /Users/vijayaraghavan.r/code/vijay/docs/clusteryamls/test/ClusterConfigWIthKonnectorAgent.yaml secret/nkp-vijay-test-cluster-29-pc-credentials created secret/nkp-vijay-test-cluster-29-pc-credentials-for-csi created secret/nkp-vijay-test-cluster-29-pc-credentials-for-konnector-agent created secret/nkp-vijay-test-cluster-29-image-registry-credentials created secret/global-nutanix-credentials created cluster.cluster.x-k8s.io/nkp-vijay-test-cluster-29 created configmap/kommander-bootstrap-configuration created secret/prism-central-metadata createdHelmChartProxy
`(devbox) vijayaraghavan.r@CQGK42CXL4 cluster-api-runtime-extensions-nutanix % kubectl get hcp konnector-agent-0199eb7b-6107-73a5-b257-aa86711fd583 -o yaml
apiVersion: addons.cluster.x-k8s.io/v1alpha1
kind: HelmChartProxy
metadata:
creationTimestamp: "2025-10-17T09:01:06Z"
finalizers:
generation: 1
name: konnector-agent-0199eb7b-6107-73a5-b257-aa86711fd583
namespace: default
ownerReferences:
kind: Cluster
name: nkp-vijay-test-cluster-29
uid: d3c523c6-f598-4760-9286-ce1e8ae8fcda
resourceVersion: "19416"
uid: 28266af0-9c08-4f52-be70-4ff7e1473afe
spec:
chartName: konnector-agent
clusterSelector:
matchLabels:
cluster.x-k8s.io/cluster-name: nkp-vijay-test-cluster-29
namespace: ntnx-system
options:
enableClientCache: false
install:
createNamespace: true
timeout: 10m0s
upgrade:
maxHistory: 10
releaseName: konnector-agent
repoURL: oci://helm-repository.default.svc/charts
tlsConfig:
caSecret:
name: helm-repository-tls
namespace: default
valuesTemplate: |-
agent:
name: konnector-agent
image:
repository: quay.io/karbon
name: k8s-agent
pc:
port: 9440
insecure: true #set this to true if PC does not have https enabled
endpoint: pc.dev.nkp.sh # eg: ip or fqdn
k8sClusterName: nkp-vijay-test-cluster-29
k8sDistribution: NKP
createSecret: false
version: 1.3.0-rc.0
status:
conditions:
status: "True"
type: Ready
status: "True"
type: HelmReleaseProxiesReady
status: "True"
type: HelmReleaseProxySpecsUpToDate
matchingClusters:
kind: Cluster
name: nkp-vijay-test-cluster-29
namespace: default
observedGeneration: 1
(devbox) vijayaraghavan.r@CQGK42CXL4 cluster-api-runtime-extensions-nutanix %`
Cluster automatically onboarded in PC
cluster deletion:
(devbox) vijayaraghavan.r@CQGK42CXL4 cluster-api-runtime-extensions-nutanix % kubectl delete -f /Users/vijayaraghavan.r/code/vijay/docs/clusteryamls/test/1konnectorAgent.yaml secret "nkp-vijay-test-cluster-29-pc-credentials" deleted secret "nkp-vijay-test-cluster-29-pc-credentials-for-csi" deleted secret "nkp-vijay-test-cluster-29-pc-credentials-for-konnector-agent" deleted secret "nkp-vijay-test-cluster-29-image-registry-credentials" deleted secret "global-nutanix-credentials" deleted cluster.cluster.x-k8s.io "nkp-vijay-test-cluster-29" deleted configmap "kommander-bootstrap-configuration" deleted secret "prism-central-metadata" deleted