diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000..59f6941b --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,232 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this +repository. + +## Overview + +The Configuration Policy Controller is a Kubernetes controller that enforces and evaluates +ConfigurationPolicy resources in Open Cluster Management. It monitors objects on managed clusters, +checks compliance against policy templates, and can automatically remediate non-compliant resources +when set to enforce mode. + +## Build, Test, and Run Commands + +### Building + +```bash +# Build the controller binary +make build + +# Build the dryrun CLI tool +make build-cmd + +# Build container image (configurable with REGISTRY, IMG, TAG env vars) +make build-images +``` + +### Testing + +```bash +# Run unit tests +make test + +# Run unit tests with coverage +make test-coverage + +# Run E2E tests (requires KinD cluster) +make e2e-test + +# Run specific E2E tests +TESTARGS="--focus=" make e2e-test + +# Setup KinD cluster for development +make kind-bootstrap-cluster-dev + +# Deploy controller to KinD and run E2E tests +make kind-tests +``` + +### Running Locally + +```bash +# Run controller locally (must set WATCH_NAMESPACE) +export WATCH_NAMESPACE= +make run + +# The controller requires a Kubernetes cluster configured via kubectl +``` + +### Linting and Formatting + +```bash +# Format code +make fmt +``` + +### Generate Manifests + +```bash +# Generate CRDs and RBAC manifests +make manifests + +# Generate DeepCopy implementations +make generate +``` + +## Architecture + +### Core Components + +**ConfigurationPolicyReconciler** (`controllers/configurationpolicy_controller.go`) + +- Main reconciler that evaluates ConfigurationPolicy resources +- Handles both `inform` (report-only) and `enforce` (remediate) modes +- Uses dynamic client to work with any Kubernetes resource type +- Supports templating with Go templates and sprig functions +- Implements watch-based evaluation for efficient resource monitoring + +**OperatorPolicyReconciler** (`controllers/operatorpolicy_controller.go`) + +- Manages OperatorPolicy resources for OLM operator lifecycle management +- Controls operator subscriptions, CSV status, and upgrade behavior + +**Evaluation Flow**: + +1. Policy is reconciled based on evaluation interval or watch events +2. Templates are resolved (if present) using go-template-utils library +3. Namespace selector determines target namespaces +4. For each object template: + - Determine desired objects (resolving selectors if needed) + - Compare with existing cluster state + - If enforce mode: create, update, or delete objects as needed + - If inform mode: report compliance status only +5. Status is updated with compliance details and related objects +6. Events are emitted to parent Policy resource + +### Key Packages + +**pkg/common** - Shared utilities: + +- `namespace_selection.go`: NamespaceSelector reconciler for efficient namespace filtering +- `common.go`: Helper functions for environment detection + +**pkg/dryrun** - CLI tool for testing policies without a cluster: + +- Simulates policy evaluation using fake clients +- Supports reading cluster resources or using local YAML files +- Provides diff output and compliance message reporting + +**pkg/mappings** - API resource mappings for the dryrun CLI + +**pkg/triggeruninstall** - Handles controller uninstallation cleanup + +### Template Processing + +The controller supports Go templating in object definitions with these special features: + +- Hub templates: can reference objects from the hub cluster (when configured) +- Context variables: `.Object`, `.ObjectName`, `.ObjectNamespace` for dynamic templating per + namespace/object +- Template functions: `fromSecret`, `fromConfigMap`, `fromClusterClaim`, `lookup`, plus sprig + functions +- `skipObject` function: allows conditional object creation based on template logic +- Encryption support: templates can include encrypted values using AES encryption + +### Compliance Types + +- **musthave**: Object must exist and match the specified fields (partial match) +- **mustnothave**: Object must not exist +- **mustonlyhave**: Object must exist and match exactly (no extra fields) + +### Watch vs Polling + +The controller supports two evaluation modes: + +- **Watch mode** (default): Uses Kubernetes watches for efficient real-time evaluation +- **Polling mode**: Periodically evaluates policies based on evaluationInterval + +The dynamic watcher (kubernetes-dependency-watches) automatically manages watches on related +objects. + +### Hosted Mode + +The controller can run in "hosted mode" where: + +- Controller runs on hub cluster +- Evaluates/enforces policies on a separate managed cluster +- Configured via `--target-kubeconfig-path` flag +- Uses separate managers for hub and managed cluster clients + +## Important Patterns + +### Object Comparison + +The comparison logic in `handleSingleKey` and `mergeSpecsHelper` is critical: + +- Merges template values into existing object to avoid false negatives +- Handles arrays specially (preserves duplicates, matches by "name" field) +- Dry-run updates verify actual API behavior before enforcement +- `zeroValueEqualsNil` parameter controls empty value handling + +### Caching and Evaluation Optimization + +- `processedPolicyCache`: Tracks evaluated objects by resourceVersion to avoid redundant comparisons +- `lastEvaluatedCache`: Prevents race conditions with controller-runtime cache staleness +- Evaluation backoff (`--evaluation-backoff`): Throttles frequent policy evaluations + +### Pruning Behavior + +When `pruneObjectBehavior` is set: + +- **DeleteIfCreated**: Removes objects created by the policy +- **DeleteAll**: Removes all objects matching the template +- Tracked via finalizers and object UIDs in status.relatedObjects + +### Dry Run CLI + +The `dryrun` command provides policy testing without cluster modification: + +```bash +# Test policy against local resources +build/_output/bin/dryrun -p policy.yaml resource1.yaml resource2.yaml + +# Test policy against live cluster (read-only) +build/_output/bin/dryrun -p policy.yaml --from-cluster + +# Compare status against expected +build/_output/bin/dryrun -p policy.yaml resources.yaml --desired-status expected-status.yaml +``` + +## Testing Guidelines + +### E2E Test Structure + +- Tests are in `test/e2e/` with descriptive case names +- Use Ginkgo/Gomega framework +- Tests can filter by label: `--label-filter='!hosted-mode'` +- Helper functions in `test/utils/utils.go` for common operations + +### Writing Tests + +- Use `utils.GetWithTimeout` for eventually-consistent checks +- Clean up resources in AfterEach blocks +- Use unique names to avoid test conflicts +- Test both inform and enforce modes where applicable + +## Configuration + +### Controller Flags + +Key flags when running the controller: + +- `--evaluation-concurrency`: Max concurrent policy evaluations (default: 2) +- `--evaluation-backoff`: Seconds before re-evaluation in watch mode (default: 10) +- `--enable-operator-policy`: Enable OperatorPolicy support +- `--target-kubeconfig-path`: Path to managed cluster kubeconfig (hosted mode) +- `--standalone-hub-templates-kubeconfig-path`: Hub cluster for template resolution + +### Environment Variables + +- `WATCH_NAMESPACE`: Namespace to monitor for policies (required when running locally) +- `POD_NAME`: Used to detect controller pod name diff --git a/build/common/Makefile.common.mk b/build/common/Makefile.common.mk index 16909ad9..1b0491c8 100755 --- a/build/common/Makefile.common.mk +++ b/build/common/Makefile.common.mk @@ -3,21 +3,26 @@ ## CLI versions (with links to the latest releases) # https://github.com/kubernetes-sigs/controller-tools/releases/latest -CONTROLLER_GEN_VERSION := v0.16.3 +CONTROLLER_GEN_VERSION := v0.19.0 # https://github.com/kubernetes-sigs/kustomize/releases/latest -KUSTOMIZE_VERSION := v5.6.0 +KUSTOMIZE_VERSION := v5.7.1 # https://github.com/golangci/golangci-lint/releases/latest GOLANGCI_VERSION := v1.64.8 # https://github.com/mvdan/gofumpt/releases/latest -GOFUMPT_VERSION := v0.7.0 +GOFUMPT_VERSION := v0.9.1 # https://github.com/daixiang0/gci/releases/latest -GCI_VERSION := v0.13.5 +GCI_VERSION := v0.13.7 # https://github.com/securego/gosec/releases/latest -GOSEC_VERSION := v2.22.2 +GOSEC_VERSION := v2.22.9 # https://github.com/kubernetes-sigs/kubebuilder/releases/latest -KBVERSION := 3.15.1 -# https://github.com/kubernetes/kubernetes/releases/latest -ENVTEST_K8S_VERSION := 1.30.x +KBVERSION := 4.9.0 +# https://github.com/alexfalkowski/gocovmerge/releases/latest +GOCOVMERGE_VERSION := v2.16.0 +# ref: https://book.kubebuilder.io/reference/envtest.html?highlight=setup-envtest#installation +# Parse the controller-runtime version from go.mod and parse to its release-X.Y git branch +ENVTEST_VERSION ?= $(shell go list -m -f "{{ .Version }}" sigs.k8s.io/controller-runtime | awk -F'[v.]' '{printf "release-%d.%d", $$2, $$3}') +# Parse the Kubernetes API version from go.mod (which is v0.Y.Z) and convert to the corresponding v1.Y.Z format +ENVTEST_K8S_VERSION := $(shell go list -m -f "{{ .Version }}" k8s.io/api | awk -F'[v.]' '{printf "1.%d", $$3}') LOCAL_BIN ?= $(error LOCAL_BIN is not set.) ifneq ($(findstring $(LOCAL_BIN), $(PATH)), $(LOCAL_BIN)) @@ -112,7 +117,8 @@ kubebuilder: .PHONY: envtest envtest: - $(call go-get-tool,sigs.k8s.io/controller-runtime/tools/setup-envtest@latest) + # Installing setup-envtest using the release-X.Y branch from the version specified in go.mod + $(call go-get-tool,sigs.k8s.io/controller-runtime/tools/setup-envtest@$(ENVTEST_VERSION)) .PHONY: gosec gosec: @@ -180,4 +186,4 @@ e2e-dependencies: GOCOVMERGE = $(LOCAL_BIN)/gocovmerge .PHONY: coverage-dependencies coverage-dependencies: - $(call go-get-tool,github.com/wadey/gocovmerge@v0.0.0-20160331181800-b5bfa59ec0ad) + $(call go-get-tool,github.com/alexfalkowski/gocovmerge/v2@$(GOCOVMERGE_VERSION)) diff --git a/controllers/configurationpolicy_controller.go b/controllers/configurationpolicy_controller.go index 04885605..fb7c44f4 100644 --- a/controllers/configurationpolicy_controller.go +++ b/controllers/configurationpolicy_controller.go @@ -1700,7 +1700,7 @@ func (r *ConfigurationPolicyReconciler) determineDesiredObjects( "expected one optional boolean argument but received %d arguments", len(skips)) } - return + return empty, err }, } @@ -2356,7 +2356,7 @@ func (r *ConfigurationPolicyReconciler) handleSingleObj( } } - return + return result, objectProperties } if exists && !obj.shouldExist { @@ -2373,7 +2373,7 @@ func (r *ConfigurationPolicyReconciler) handleSingleObj( result.events = append(result.events, objectTmplEvalEvent{false, reasonWantNotFoundExists, ""}) } - return + return result, objectProperties } if !exists && !obj.shouldExist { @@ -2381,7 +2381,7 @@ func (r *ConfigurationPolicyReconciler) handleSingleObj( // it is a must not have and it does not exist, so it is compliant result.events = append(result.events, objectTmplEvalEvent{true, reasonWantNotFoundDNE, ""}) - return + return result, objectProperties } // object exists and the template requires it, so we need to check specific fields to see if we have a match @@ -2458,7 +2458,7 @@ func (r *ConfigurationPolicyReconciler) handleSingleObj( } } - return + return result, objectProperties } // getMapping takes in a raw object, decodes it, and maps it to an existing group/kind @@ -3565,7 +3565,7 @@ func handleKeys( } } - return + return throwSpecViolation, message, updateNeeded, statusMismatch, missingKey } func removeFieldsForComparison(obj *unstructured.Unstructured) { diff --git a/controllers/configurationpolicy_utils.go b/controllers/configurationpolicy_utils.go index 7d69f9f8..bae606e2 100644 --- a/controllers/configurationpolicy_utils.go +++ b/controllers/configurationpolicy_utils.go @@ -617,7 +617,7 @@ func createStatus( compliancyDetailsMsg = getCombinedCompliancyDetailsMsg(msgMap, resourceName, compliancyDetailsMsg) } - return + return compliant, compliancyDetailsReason, compliancyDetailsMsg } func setCompliancyDetailsMsgEnd(compliancyDetailsMsg string) string { diff --git a/deploy/crds/kustomize_configurationpolicy/policy.open-cluster-management.io_configurationpolicies.yaml b/deploy/crds/kustomize_configurationpolicy/policy.open-cluster-management.io_configurationpolicies.yaml index 315e3a86..66556c37 100644 --- a/deploy/crds/kustomize_configurationpolicy/policy.open-cluster-management.io_configurationpolicies.yaml +++ b/deploy/crds/kustomize_configurationpolicy/policy.open-cluster-management.io_configurationpolicies.yaml @@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.16.3 + controller-gen.kubebuilder.io/version: v0.19.0 name: configurationpolicies.policy.open-cluster-management.io spec: group: policy.open-cluster-management.io diff --git a/deploy/crds/kustomize_operatorpolicy/policy.open-cluster-management.io_operatorpolicies.yaml b/deploy/crds/kustomize_operatorpolicy/policy.open-cluster-management.io_operatorpolicies.yaml index f842ae70..e02f4a76 100644 --- a/deploy/crds/kustomize_operatorpolicy/policy.open-cluster-management.io_operatorpolicies.yaml +++ b/deploy/crds/kustomize_operatorpolicy/policy.open-cluster-management.io_operatorpolicies.yaml @@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.16.3 + controller-gen.kubebuilder.io/version: v0.19.0 name: operatorpolicies.policy.open-cluster-management.io spec: group: policy.open-cluster-management.io diff --git a/deploy/crds/policy.open-cluster-management.io_configurationpolicies.yaml b/deploy/crds/policy.open-cluster-management.io_configurationpolicies.yaml index 7b6e1a43..f37bc10f 100644 --- a/deploy/crds/policy.open-cluster-management.io_configurationpolicies.yaml +++ b/deploy/crds/policy.open-cluster-management.io_configurationpolicies.yaml @@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.16.3 + controller-gen.kubebuilder.io/version: v0.19.0 labels: policy.open-cluster-management.io/policy-type: template name: configurationpolicies.policy.open-cluster-management.io diff --git a/deploy/crds/policy.open-cluster-management.io_operatorpolicies.yaml b/deploy/crds/policy.open-cluster-management.io_operatorpolicies.yaml index 7026a80f..b56875cf 100644 --- a/deploy/crds/policy.open-cluster-management.io_operatorpolicies.yaml +++ b/deploy/crds/policy.open-cluster-management.io_operatorpolicies.yaml @@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.16.3 + controller-gen.kubebuilder.io/version: v0.19.0 labels: policy.open-cluster-management.io/policy-type: template name: operatorpolicies.policy.open-cluster-management.io diff --git a/pkg/dryrun/policy.open-cluster-management.io_configurationpolicies.yaml b/pkg/dryrun/policy.open-cluster-management.io_configurationpolicies.yaml index 7b6e1a43..f37bc10f 100644 --- a/pkg/dryrun/policy.open-cluster-management.io_configurationpolicies.yaml +++ b/pkg/dryrun/policy.open-cluster-management.io_configurationpolicies.yaml @@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - controller-gen.kubebuilder.io/version: v0.16.3 + controller-gen.kubebuilder.io/version: v0.19.0 labels: policy.open-cluster-management.io/policy-type: template name: configurationpolicies.policy.open-cluster-management.io