feat: Add support for deploying K8s Agent as Helm Addon during cluster deployment (#1345)

**What problem does this PR solve?**:
This PR introduces support for deploying the Nutanix K8s Agent as a
Helm-based addon as part of the cluster deployment workflow in the CAREN
repository.

Key changes:

Added support for HelmAddon strategy in the K8s Registration Agent
handler.

Updated handler logic to fetch and apply Helm chart configuration
dynamically using the provided credentials.

Created unit tests to validate HelmAddon flow and error scenarios.

Ensured the agent’s credentials are handled securely through Kubernetes
Secrets instead of directly passing via values.yaml.

This enhancement enables seamless installation of the K8s Agent via
HelmReleaseProxy (HRP) and HelmChartProxy (HCP) without manual
post-deployment intervention.

**Motivation**

Previously, the K8s Agent installation was not integrated into the
cluster creation process and required manual setup.
By integrating it as an addon:

The K8s Agent will be automatically deployed during cluster
provisioning.

Sensitive credentials are now secured using Kubernetes Secrets.

This aligns the agent deployment with other addon-based lifecycle
management mechanisms.
**Which issue(s) this PR fixes**:
Fixes #

**How Has This Been Tested?**:
Verified K8s Agent deployment using the HelmAddon strategy.
Confirmed that:

Pre-install hooks execute as expected.

The secret is created and retained after installation.

Post-upgrade hooks are triggered correctly.

Added unit tests under k8sregistrationagent/handler_test.go

[K8s agent
documentation](https://docs.google.com/document/d/1E8qF94EYRX12yCpWDK_NqHHvH5tjqAtrD-DQDwMK7kU/edit?tab=t.0#heading=h.f3fcu3k7nqm)
**Special notes for your reviewer**:
<!--
Use this to provide any additional information to the reviewers.
This may include:
- Best way to review the PR.
- Where the author wants the most review attention on.
- etc.
-->

Cluster class used for testing with konnector Agent:

[ClusterConfigWIthKonnectorAgent.yaml](https://github.com/user-attachments/files/22967186/ClusterConfigWIthKonnectorAgent.yaml)

Cluster creation - with konnector agent secret and addon:
`(devbox) vijayaraghavan.r@CQGK42CXL4
cluster-api-runtime-extensions-nutanix % kubectl apply -f
/Users/vijayaraghavan.r/code/vijay/docs/clusteryamls/test/ClusterConfigWIthKonnectorAgent.yaml
secret/nkp-vijay-test-cluster-29-pc-credentials created
secret/nkp-vijay-test-cluster-29-pc-credentials-for-csi created
secret/nkp-vijay-test-cluster-29-pc-credentials-for-konnector-agent
created
secret/nkp-vijay-test-cluster-29-image-registry-credentials created
secret/global-nutanix-credentials created
cluster.cluster.x-k8s.io/nkp-vijay-test-cluster-29 created
configmap/kommander-bootstrap-configuration created
secret/prism-central-metadata created`

HelmChartProxy
`(devbox) vijayaraghavan.r@CQGK42CXL4
cluster-api-runtime-extensions-nutanix % kubectl get hcp
konnector-agent-0199eb7b-6107-73a5-b257-aa86711fd583 -o yaml
apiVersion: addons.cluster.x-k8s.io/v1alpha1
kind: HelmChartProxy
metadata:
  creationTimestamp: "2025-10-17T09:01:06Z"
  finalizers:
  - helmchartproxy.addons.cluster.x-k8s.io
  generation: 1
  name: konnector-agent-0199eb7b-6107-73a5-b257-aa86711fd583
  namespace: default
  ownerReferences:
  - apiVersion: cluster.x-k8s.io/v1beta1
    kind: Cluster
    name: nkp-vijay-test-cluster-29
    uid: d3c523c6-f598-4760-9286-ce1e8ae8fcda
  resourceVersion: "19416"
  uid: 28266af0-9c08-4f52-be70-4ff7e1473afe
spec:
  chartName: konnector-agent
  clusterSelector:
    matchLabels:
      cluster.x-k8s.io/cluster-name: nkp-vijay-test-cluster-29
  namespace: ntnx-system
  options:
    enableClientCache: false
    install:
      createNamespace: true
    timeout: 10m0s
    upgrade:
      maxHistory: 10
  releaseName: konnector-agent
  repoURL: oci://helm-repository.default.svc/charts
  tlsConfig:
    caSecret:
      name: helm-repository-tls
      namespace: default
  valuesTemplate: |-
    agent:
      name: konnector-agent
      image:
        repository: quay.io/karbon
        name: k8s-agent
    pc:
      port: 9440
      insecure: true #set this to true if PC does not have https enabled
      endpoint: pc.dev.nkp.sh # eg: ip or fqdn
    k8sClusterName: nkp-vijay-test-cluster-29
    k8sDistribution: NKP
    createSecret: false
  version: 1.3.0-rc.0
status:
  conditions:
  - lastTransitionTime: "2025-10-17T09:03:25Z"
    status: "True"
    type: Ready
  - lastTransitionTime: "2025-10-17T09:03:25Z"
    status: "True"
    type: HelmReleaseProxiesReady
  - lastTransitionTime: "2025-10-17T09:01:06Z"
    status: "True"
    type: HelmReleaseProxySpecsUpToDate
  matchingClusters:
  - apiVersion: cluster.x-k8s.io/v1beta1
    kind: Cluster
    name: nkp-vijay-test-cluster-29
    namespace: default
  observedGeneration: 1
(devbox) vijayaraghavan.r@CQGK42CXL4
cluster-api-runtime-extensions-nutanix %`

Cluster automatically onboarded in PC
<img width="1649" height="79" alt="Screenshot 2025-10-17 at 2 35 45 PM"
src="https://github.com/user-attachments/assets/57e0192e-7726-4983-a14e-2ed5dc9680f1"
/>

cluster deletion:
`(devbox) vijayaraghavan.r@CQGK42CXL4
cluster-api-runtime-extensions-nutanix % kubectl delete -f
/Users/vijayaraghavan.r/code/vijay/docs/clusteryamls/test/1konnectorAgent.yaml
secret "nkp-vijay-test-cluster-29-pc-credentials" deleted
secret "nkp-vijay-test-cluster-29-pc-credentials-for-csi" deleted
secret "nkp-vijay-test-cluster-29-pc-credentials-for-konnector-agent"
deleted
secret "nkp-vijay-test-cluster-29-image-registry-credentials" deleted
secret "global-nutanix-credentials" deleted
cluster.cluster.x-k8s.io "nkp-vijay-test-cluster-29" deleted
configmap "kommander-bootstrap-configuration" deleted
secret "prism-central-metadata" deleted`

<img width="691" height="252" alt="Screenshot 2025-10-17 at 2 39 23 PM"
src="https://github.com/user-attachments/assets/f482e575-78f6-4ad0-ac49-2835a555e3f9"
/>

---------

Co-authored-by: Manoj Surudwad <[email protected]>
Co-authored-by: Dimitri Koshkin <[email protected]>

Author: 3 people

api/v1alpha1/addon_types.go

@@ -90,6 +90,9 @@ type NutanixAddons struct {
 
 	// +kubebuilder:validation:Optional
 	COSI *NutanixCOSI `json:"cosi,omitempty"`
+
+	// +kubebuilder:validation:Optional
+	KonnectorAgent *NutanixKonnectorAgent `json:"konnectorAgent,omitempty"`
 }
 
 type GenericAddons struct {
@@ -371,3 +374,15 @@ type Ingress struct {
 	// +kubebuilder:validation:Enum="aws-lb-controller"
 	Provider string `json:"provider"`
 }
+
+type NutanixKonnectorAgent struct {
+	// A reference to the Secret for credential information for the target Prism Central instance
+	// +kubebuilder:validation:Optional
+	Credentials *NutanixKonnectorAgentCredentials `json:"credentials,omitempty"`
+}
+
+type NutanixKonnectorAgentCredentials struct {
+	// A reference to the Secret containing the credentials used by the Konnector agent.
+	// +kubebuilder:validation:Required
+	SecretRef LocalObjectReference `json:"secretRef"`
+}

api/v1alpha1/constants.go

@@ -32,6 +32,8 @@ const (
 	ServiceLoadBalancerVariableName = "serviceLoadBalancer"
 	// RegistryAddonVariableName is the OCI registry config patch variable name.
 	RegistryAddonVariableName = "registry"
+	// KonnectorAgentVariableName is the Nutanix konnector-agent addon config patch variable name.
+	KonnectorAgentVariableName = "konnectorAgent"
 
 	// GlobalMirrorVariableName is the global image registry mirror patch variable name.
 	GlobalMirrorVariableName = "globalImageRegistryMirror"

api/v1alpha1/crds/caren.nutanix.com_nutanixclusterconfigs.yaml

@@ -235,6 +235,28 @@ spec:
                         - defaultStorage
                         - providers
                       type: object
+                    konnectorAgent:
+                      properties:
+                        credentials:
+                          description: A reference to the Secret for credential information for the target Prism Central instance
+                          properties:
+                            secretRef:
+                              description: A reference to the Secret containing the credentials used by the Konnector agent.
+                              properties:
+                                name:
+                                  description: |-
+                                    Name of the referent.
+                                    More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+                                  maxLength: 253
+                                  minLength: 1
+                                  type: string
+                              required:
+                                - name
+                              type: object
+                          required:
+                            - secretRef
+                          type: object
+                      type: object
                     nfd:
                       description: NFD tells us to enable or disable the node feature discovery addon.
                       properties:

api/v1alpha1/zz_generated.deepcopy.go

@@ -68,6 +68,12 @@ type Addons struct {
 	COSI *COSI `json:"cosi,omitempty"`
 
 	Ingress *Ingress `json:"ingress,omitempty"`
+
+	NutanixKonnectorAgent *NutanixKonnectorAgent `json:"konnectorAgent,omitempty"`
+}
+
+type NutanixKonnectorAgent struct {
+	carenv1.NutanixKonnectorAgent `json:",inline"`
 }
 
 type CSI struct {

api/variables/aggregate_types.go

@@ -92,6 +92,8 @@ A Helm chart for cluster-api-runtime-extensions-nutanix
 | hooks.csi.snapshot-controller.helmAddonStrategy.defaultValueTemplateConfigMap.name | string | `"default-snapshot-controller-helm-values-template"` |  |
 | hooks.ingress.awsLoadBalancerController.defaultValueTemplateConfigMap.create | bool | `true` |  |
 | hooks.ingress.awsLoadBalancerController.defaultValueTemplateConfigMap.name | string | `"default-aws-load-balancer-controller-helm-values-template"` |  |
+| hooks.konnectorAgent.helmAddonStrategy.defaultValueTemplateConfigMap.create | bool | `true` |  |
+| hooks.konnectorAgent.helmAddonStrategy.defaultValueTemplateConfigMap.name | string | `"default-konnector-agent-helm-values-template"` |  |
 | hooks.nfd.crsStrategy.defaultInstallationConfigMap.name | string | `"node-feature-discovery"` |  |
 | hooks.nfd.helmAddonStrategy.defaultValueTemplateConfigMap.create | bool | `true` |  |
 | hooks.nfd.helmAddonStrategy.defaultValueTemplateConfigMap.name | string | `"default-nfd-helm-values-template"` |  |

charts/cluster-api-runtime-extensions-nutanix/README.md

@@ -0,0 +1,12 @@
+agent:
+  name: {{ .AgentName }}
+  image:
+    repository: quay.io/karbon
+    name: k8s-agent
+pc:
+  port: {{ .PrismCentralPort }}
+  insecure: {{ .PrismCentralInsecure }} #set this to true if PC does not have https enabled
+  endpoint: {{ .PrismCentralHost }} # eg: ip or fqdn
+k8sClusterName: {{ .ClusterName }}
+k8sDistribution: NKP
+createSecret: false

charts/cluster-api-runtime-extensions-nutanix/addons/konnector-agent/values-template.yaml

@@ -45,6 +45,7 @@ spec:
         - --csi.snapshot-controller.helm-addon.default-values-template-configmap-name={{ (index .Values.hooks.csi "snapshot-controller").helmAddonStrategy.defaultValueTemplateConfigMap.name }}
         - --ccm.aws.helm-addon.default-values-template-configmap-name={{ .Values.hooks.ccm.aws.helmAddonStrategy.defaultValueTemplateConfigMap.name }}
         - --cosi.controller.helm-addon.default-values-template-configmap-name={{ .Values.hooks.cosi.controller.helmAddonStrategy.defaultValueTemplateConfigMap.name }}
+        - --konnector-agent.helm-addon.default-values-template-configmap-name={{ .Values.hooks.konnectorAgent.helmAddonStrategy.defaultValueTemplateConfigMap.name }}
         {{- range $k, $v := .Values.hooks.ccm.aws.k8sMinorVersionToCCMVersion }}
         - --ccm.aws.aws-ccm-versions={{ $k }}={{ $v }}
         {{- end }}

charts/cluster-api-runtime-extensions-nutanix/templates/deployment.yaml

@@ -35,6 +35,10 @@ data:
     ChartName: cosi
     ChartVersion: 0.0.1-alpha.5
     RepositoryURL: '{{ if .Values.helmRepository.enabled }}oci://helm-repository.{{ .Release.Namespace }}.svc/charts{{ else }}https://mesosphere.github.io/charts/stable/{{ end }}'
+  konnector-agent: |
+    ChartName: konnector-agent
+    ChartVersion: 1.3.0-rc.1
+    RepositoryURL: '{{ if .Values.helmRepository.enabled }}oci://helm-repository.{{ .Release.Namespace }}.svc/charts{{ else }}https://mesosphere.github.io/charts/stable{{ end }}'
   local-path-provisioner-csi: |
     ChartName: local-path-provisioner
     ChartVersion: 0.0.32

charts/cluster-api-runtime-extensions-nutanix/templates/helm-config.yaml

@@ -0,0 +1,12 @@
+# Copyright 2025 Nutanix. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+{{- if .Values.hooks.konnectorAgent.helmAddonStrategy.defaultValueTemplateConfigMap.name }}
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: '{{ .Values.hooks.konnectorAgent.helmAddonStrategy.defaultValueTemplateConfigMap.name }}'
+data:
+  values.yaml: |-
+    {{- .Files.Get "addons/konnector-agent/values-template.yaml" | nindent 4 }}
+{{- end -}}