|
| 1 | +# Addon Configuration using FleetConfig Controller |
| 2 | + |
| 3 | +## Overview |
| 4 | + |
| 5 | +FleetConfig Controller (FCC) provides a declarative wrapper around the Open Cluster Management (OCM) [addon framework](https://open-cluster-management.io/docs/concepts/add-on-extensibility/addon/), simplifying the process of creating and deploying custom addons across your fleet of managed clusters. Instead of manually running `clusteradm` commands, you can define addons in your Hub and Spoke resources, and FCC handles the lifecycle management automatically. |
| 6 | + |
| 7 | +The addon framework in FCC supports two types of addons: |
| 8 | +1. **Custom Addons** - User-defined addons configured via `AddOnConfig` in the Hub spec |
| 9 | +2. **Built-in Hub Addons** - Pre-configured addons (ArgoCD and Governance Policy Framework) via `HubAddOn` |
| 10 | + |
| 11 | +## Custom Addons |
| 12 | + |
| 13 | +### How It Works |
| 14 | + |
| 15 | +FCC wraps the `clusteradm addon create` command, allowing you to define custom addon templates declaratively. When you specify an `AddOnConfig` in your Hub resource, FCC: |
| 16 | + |
| 17 | +1. Looks for a ConfigMap containing the addon manifests |
| 18 | +2. Creates an `AddOnTemplate` resource in the hub cluster using clusteradm |
| 19 | +3. Makes the addon available for installation on spoke clusters |
| 20 | +4. Manages the lifecycle (create/update/delete) based on your configuration |
| 21 | + |
| 22 | +### Configuring a Custom Addon |
| 23 | + |
| 24 | +#### Step 1: Create a ConfigMap with Addon Manifests |
| 25 | + |
| 26 | +First, create a ConfigMap containing your addon manifests. The ConfigMap must be named following the pattern: `fleet-addon-<name>-<version>` |
| 27 | + |
| 28 | +```yaml |
| 29 | +apiVersion: v1 |
| 30 | +kind: ConfigMap |
| 31 | +metadata: |
| 32 | + name: fleet-addon-myapp-v1.0.0 |
| 33 | + namespace: <hub-namespace> |
| 34 | +data: |
| 35 | + # Option 1: Provide raw manifests directly |
| 36 | + manifestsRaw: | |
| 37 | + apiVersion: apps/v1 |
| 38 | + kind: Deployment |
| 39 | + metadata: |
| 40 | + name: myapp-agent |
| 41 | + namespace: open-cluster-management-agent-addon |
| 42 | + spec: |
| 43 | + # ... your deployment spec |
| 44 | + --- |
| 45 | + # Additional resources... |
| 46 | + |
| 47 | + # Option 2: Provide a URL to manifests (mutually exclusive with manifestsRaw) |
| 48 | + # manifestsURL: "https://example.com/myapp/manifests.yaml" |
| 49 | +``` |
| 50 | + |
| 51 | +**Note**: You must provide either `manifestsRaw` or `manifestsURL`, but not both. |
| 52 | + |
| 53 | +#### Step 2: Define AddOnConfig in Hub Resource |
| 54 | + |
| 55 | +Add the addon configuration to your Hub resource: |
| 56 | + |
| 57 | +```yaml |
| 58 | +apiVersion: fleet.ocm.io/v1beta1 |
| 59 | +kind: Hub |
| 60 | +metadata: |
| 61 | + name: my-hub |
| 62 | +spec: |
| 63 | + # ... other hub configuration |
| 64 | + |
| 65 | + addOnConfigs: |
| 66 | + - name: myapp |
| 67 | + version: v1.0.0 |
| 68 | + |
| 69 | + # Optional: Enable hub registration for the addon agent |
| 70 | + # Allows the addon agent to register and communicate with the hub |
| 71 | + hubRegistration: false |
| 72 | + |
| 73 | + # Optional: Overwrite if addon already exists |
| 74 | + overwrite: false |
| 75 | + |
| 76 | + # Optional: ClusterRole binding for addon agent in the cluster namespace on the Hub cluster |
| 77 | + clusterRoleBinding: "cluster-admin" |
| 78 | +``` |
| 79 | +
|
| 80 | +#### AddOnConfig Fields |
| 81 | +
|
| 82 | +- **name** (required): The name of the addon |
| 83 | +- **version** (optional): The addon version, defaults to "v0.0.1" |
| 84 | +- **clusterRoleBinding** (optional): RoleBinding to a ClusterRole in the cluster namespace for the addon agent |
| 85 | +- **hubRegistration** (optional): Enable agent registration to the hub cluster (default: false) |
| 86 | +- **overwrite** (optional): Overwrite the addon if it already exists (default: false) |
| 87 | +
|
| 88 | +### Enabling Custom Addons on Spoke Clusters |
| 89 | +
|
| 90 | +Once you've defined an `AddOnConfig` in your Hub, you can enable it on specific spoke clusters by referencing it in the Spoke resource: |
| 91 | + |
| 92 | +```yaml |
| 93 | +apiVersion: fleet.ocm.io/v1beta1 |
| 94 | +kind: Spoke |
| 95 | +metadata: |
| 96 | + name: my-spoke |
| 97 | +spec: |
| 98 | + # ... other spoke configuration |
| 99 | + |
| 100 | + addOns: |
| 101 | + - configName: myapp # Must match an AddOnConfig.name or HubAddOn.name |
| 102 | + |
| 103 | + # Optional: Add annotations to the addon |
| 104 | + annotations: |
| 105 | + example.com/type: "custom" |
| 106 | + |
| 107 | + # Optional: Provide deployment configuration |
| 108 | + deploymentConfig: |
| 109 | + nodePlacement: |
| 110 | + nodeSelector: |
| 111 | + node-role.kubernetes.io/worker: "" |
| 112 | + customizedVariables: |
| 113 | + - name: IMAGE |
| 114 | + value: "quay.io/myorg/myapp:v1.0.0" |
| 115 | +``` |
| 116 | + |
| 117 | +#### AddOn Fields |
| 118 | + |
| 119 | +- **configName** (required): The name of the addon, must match an `AddOnConfig.name` or `HubAddOn.name` |
| 120 | +- **annotations** (optional): Annotations to apply to the addon |
| 121 | +- **deploymentConfig** (optional): Additional configuration for addon deployment, creates an `AddOnDeploymentConfig` resource |
| 122 | + |
| 123 | +The `deploymentConfig` field accepts the full [AddOnDeploymentConfigSpec](https://github.com/open-cluster-management-io/api/blob/main/addon/v1alpha1/types_addondeploymentconfig.go) from OCM, allowing you to configure: |
| 124 | +- Node placement (node selectors, tolerations) |
| 125 | +- Replicas and availability policy |
| 126 | +- Custom variables for manifest templating |
| 127 | +- Resource requirements |
| 128 | + |
| 129 | +## Built-in Hub Addons |
| 130 | + |
| 131 | +FCC includes two built-in addons that can be easily enabled without requiring a ConfigMap: |
| 132 | + |
| 133 | +1. **argocd** - ArgoCD GitOps controller |
| 134 | +2. **governance-policy-framework** - OCM governance policy framework |
| 135 | + |
| 136 | +### Configuring Built-in Hub Addons |
| 137 | + |
| 138 | +Add `HubAddOn` entries to your Hub resource: |
| 139 | + |
| 140 | +```yaml |
| 141 | +apiVersion: fleet.ocm.io/v1beta1 |
| 142 | +kind: Hub |
| 143 | +metadata: |
| 144 | + name: my-hub |
| 145 | +spec: |
| 146 | + # ... other hub configuration |
| 147 | + |
| 148 | + hubAddOns: |
| 149 | + - name: argocd |
| 150 | + createNamespace: true |
| 151 | + |
| 152 | + - name: governance-policy-framework |
| 153 | + installNamespace: open-cluster-management-addon |
| 154 | + createNamespace: false |
| 155 | +``` |
| 156 | + |
| 157 | +#### HubAddOn Fields |
| 158 | + |
| 159 | +- **name** (required): The name of the built-in addon. Must be one of: `argocd`, `governance-policy-framework` |
| 160 | +- **installNamespace** (optional): The namespace to install the addon in. Defaults to "open-cluster-management-addon". Not supported for `argocd`. |
| 161 | +- **createNamespace** (optional): Whether to create the namespace if it doesn't exist (default: false) |
| 162 | + |
| 163 | +### Enabling Built-in Addons on Spoke Clusters |
| 164 | + |
| 165 | +Built-in hub addons can be enabled on spoke clusters the same way as custom addons: |
| 166 | + |
| 167 | +```yaml |
| 168 | +apiVersion: fleet.ocm.io/v1beta1 |
| 169 | +kind: Spoke |
| 170 | +metadata: |
| 171 | + name: my-spoke |
| 172 | +spec: |
| 173 | + addOns: |
| 174 | + - configName: argocd |
| 175 | + - configName: governance-policy-framework |
| 176 | +``` |
| 177 | + |
| 178 | +## Implementation Details |
| 179 | + |
| 180 | +The addon creation logic in FCC (found in `internal/controller/v1beta1/addon.go`) performs the following operations: |
| 181 | + |
| 182 | +### For Custom Addons (`AddOnConfig`) |
| 183 | + |
| 184 | +1. **List existing addons**: Queries for `AddOnTemplate` resources with the `addon.open-cluster-management.io/managedBy` label |
| 185 | +2. **Compare requested vs created**: Determines which addons need to be created or deleted |
| 186 | +3. **Delete obsolete addons**: Removes addons no longer in the spec |
| 187 | +4. **Create new addons**: For each new addon: |
| 188 | + - Loads the ConfigMap containing manifests (`fleet-addon-<name>-<version>`) |
| 189 | + - Validates manifest configuration (either raw or URL) |
| 190 | + - Constructs the `clusteradm addon create` command with appropriate flags |
| 191 | + - Executes the command to create the `AddOnTemplate` |
| 192 | + - Applies the `addon.open-cluster-management.io/managedBy` label for lifecycle tracking |
| 193 | + |
| 194 | +### Command Construction |
| 195 | + |
| 196 | +The controller builds `clusteradm` commands like: |
| 197 | + |
| 198 | +```bash |
| 199 | +clusteradm addon create <name> \ |
| 200 | + --version=<version> \ |
| 201 | + --labels=addon.open-cluster-management.io/managedBy=fleetconfig-controller \ |
| 202 | + --filename=<manifests-path-or-url> \ |
| 203 | + [--hub-registration] \ |
| 204 | + [--overwrite] \ |
| 205 | + [--cluster-role-bind=<role>] \ |
| 206 | + --kubeconfig=<path> \ |
| 207 | + --context=<context> |
| 208 | +``` |
| 209 | + |
| 210 | +### For Built-in Hub Addons |
| 211 | + |
| 212 | +Built-in hub addons follow a similar pattern but use pre-configured manifest sources from the `clusteradm` tool itself. |
| 213 | + |
| 214 | +## Best Practices |
| 215 | + |
| 216 | +1. **Version Management**: Always specify explicit versions for your custom addons to ensure predictable deployments. [Semantic versioning](https://semver.org/) is recommended, but not required. |
| 217 | +2. **ConfigMap Naming**: Follow the naming convention strictly: `fleet-addon-<name>-<version>` |
| 218 | +3. **Manifest Sources**: Use `manifestsURL` for large manifests or those stored in git; use `manifestsRaw` for simple, small manifests |
| 219 | +4. **Hub Registration**: Only enable `hubRegistration` when your addon agent needs to communicate back to the hub |
| 220 | +5. **Namespace Management**: For built-in hub addons, set `createNamespace: true` if you're unsure whether the namespace exists |
| 221 | +6. **Labels**: FCC automatically applies the `addon.open-cluster-management.io/managedBy` label to track resources it manages |
| 222 | +7. **Cleanup**: When removing an addon from the Hub spec, FCC will automatically delete the corresponding `AddOnTemplate` |
| 223 | + |
| 224 | +## Troubleshooting |
| 225 | + |
| 226 | +### Addon Not Created |
| 227 | + |
| 228 | +- Verify the ConfigMap exists with the correct name format |
| 229 | +- Check the Hub controller logs for clusteradm command errors |
| 230 | +- Ensure the manifest source (raw or URL) is valid |
| 231 | + |
| 232 | +### Addon Not Installing on Spoke |
| 233 | + |
| 234 | +- Confirm the `configName` matches an existing `AddOnConfig` or `HubAddOn` name |
| 235 | +- Check the spoke controller logs for addon reconciliation errors |
| 236 | +- Verify the addon template exists in the hub cluster |
| 237 | + |
| 238 | +### Version Conflicts |
| 239 | + |
| 240 | +- If you need to update an addon version, create a new ConfigMap with the new version |
| 241 | +- Update the `AddOnConfig` version in the Hub spec |
| 242 | +- The old addon template will be automatically removed |
| 243 | + |
0 commit comments