Add YAML merge strategy to copy operations

Signed-off-by: Stefan Prodan <[email protected]>

Author: stefanprodan

README.md

@@ -30,7 +30,7 @@ apiVersion: source.extensions.fluxcd.io/v1beta1
 kind: ArtifactGenerator
 metadata:
   name: my-app
-  namespace: flux-system
+  namespace: apps
 spec:
   sources:
     - alias: backend
@@ -56,9 +56,9 @@ apiVersion: kustomize.toolkit.fluxcd.io/v1
 kind: Kustomization
 metadata:
   name: my-app
-  namespace: flux-system
+  namespace: apps
 spec:
-  interval: 10m
+  interval: 15m
   sourceRef:
     kind: ExternalArtifact
     name: my-app-composite
@@ -75,26 +75,27 @@ apiVersion: source.extensions.fluxcd.io/v1beta1
 kind: ArtifactGenerator
 metadata:
   name: platform-services
-  namespace: flux-system
+  namespace: platform
 spec:
   sources:
     - alias: monorepo
       kind: GitRepository
       name: platform-monorepo
   artifacts:
-    - name: policy-config
+    - name: policies
       copy:
         - from: "@monorepo/platform/policy/**"
           to: "@artifact/"
     - name: auth-service
       copy:
         - from: "@monorepo/services/auth/deploy/**"
           to: "@artifact/"
+          exclude: ["*.md"]
     - name: api-gateway
       copy:
         - from: "@monorepo/services/gateway/deploy/**"
           to: "@artifact/"
-          exclude: ["**/charts/**"]
+          exclude: ["*.md", "**/charts/**"]
 ```
 
 Each service gets its own ExternalArtifact with an independent revision.
@@ -116,43 +117,56 @@ spec:
     - alias: chart
       kind: OCIRepository
       name: podinfo-chart
-      namespace: apps
     - alias: repo
       kind: GitRepository
       name: podinfo-values
-      namespace: apps
   artifacts:
     - name: podinfo-composite
       originRevision: "@chart" # Track chart version
       copy:
         - from: "@chart/"
           to: "@artifact/"
+        - from: "@repo/charts/podinfo/values.yaml"
+          to: "@artifact/podinfo/values.yaml"
+          strategy: Overwrite
         - from: "@repo/charts/podinfo/values-prod.yaml"
-          to: "@artifact/podinfo/values.yaml" # Override default values
+          to: "@artifact/podinfo/values.yaml"
+          strategy: Merge
+---
+apiVersion: helm.toolkit.fluxcd.io/v2
+kind: HelmRelease
+metadata:
+  name: podinfo
+  namespace: apps
+spec:
+  interval: 15m
+  releaseName: podinfo
+  chartRef:
+    kind: ExternalArtifact
+    name: podinfo-composite
 ```
 
 ## Key Features
 
 ### Content-based Revision Tracking
 
-The generated artifacts are versioned as `latest@sha256:<hash>` where the hash
-is computed from the artifact's content. This means:
+The generated artifacts are versioned based on the hash computed from the artifact's content.
+This means:
 
 - ✅ Identical content = same revision (no unnecessary reconciliations)
-- ✅ Any change = new revision (guaranteed updates)
+- ✅ Any change to included content = new revision (guaranteed updates)
 - ✅ Automatic change detection across all source types
 
-The controller attaches the origin source revision (e.g. Git commit SHA)
-to the generated artifact metadata for traceability and Flux commit status updates.
-
 ### Flexible Copy Operations
 
-Copy operations support glob patterns and exclusions:
+Copy operations support glob patterns, exclusions and YAML merge:
 
 - `@source/file.yaml` → `@artifact/dest/` - Copy file into directory
 - `@source/dir/` → `@artifact/dest/` - Copy directory as subdirectory
 - `@source/dir/**` → `@artifact/dest/` - Copy directory contents
 - `@source/file.yaml` → `@artifact/existing.yaml` - Later copy overwrites earlier ones
+- `exclude: ["*.md"]` - Exclude matching files from copy
+- `strategy: Merge` - Deep merge YAML files instead of overwriting
 
 ## Use Cases
 
@@ -164,7 +178,7 @@ Copy operations support glob patterns and exclusions:
 - **Vendor Chart Customization** - Combine upstream Helm charts from OCI registries with
   your organization's custom values and configuration overrides stored in Git.
 - **Selective Deployment** - Deploy only changed components from large repositories
-  by decomposing them into focused artifacts.
+  by decomposing them into dedicated artifacts.
 
 ## API Reference
 
@@ -173,4 +187,4 @@ Copy operations support glob patterns and exclusions:
 ## Contributing
 
 This project is Apache 2.0 licensed and accepts contributions via GitHub pull requests.
-To start contributing please see the [development guide](CONTRIBUTING.md).
+To start contributing please see the [contrib guide](CONTRIBUTING.md).

api/v1beta1/artifactgenerator_types.go

@@ -35,6 +35,8 @@ const (
 	AccessDeniedReason               = "AccessDenied"
 	ValidationFailedReason           = "ValidationFailed"
 	SourceFetchFailedReason          = "SourceFetchFailed"
+	OverwriteStrategy                = "Overwrite"
+	MergeStrategy                    = "Merge"
 	EnabledValue                     = "enabled"
 	DisabledValue                    = "disabled"
 )
@@ -143,6 +145,14 @@ type CopyOperation struct {
 	// +kubebuilder:validation:MaxItems=100
 	// +optional
 	Exclude []string `json:"exclude,omitempty"`
+
+	// Strategy specifies the copy strategy to use.
+	// 'Overwrite' will overwrite existing files in the destination.
+	// 'Merge' is for merging YAML files using Helm values merge strategy.
+	// If not specified, defaults to 'Overwrite'.
+	// +optional
+	// +kubebuilder:validation:Enum=Overwrite;Merge
+	Strategy string `json:"strategy,omitempty"`
 }
 
 // ArtifactGeneratorStatus defines the observed state of ArtifactGenerator.

config/crd/bases/source.extensions.fluxcd.io_artifactgenerators.yaml

@@ -68,6 +68,16 @@ spec:
                             maxLength: 1024
                             pattern: ^@([a-z0-9]([a-z0-9_-]*[a-z0-9])?)/(.*)$
                             type: string
+                          strategy:
+                            description: |-
+                              Strategy specifies the copy strategy to use.
+                              'Overwrite' will overwrite existing files in the destination.
+                              'Merge' is for merging YAML files using Helm values merge strategy.
+                              If not specified, defaults to 'Overwrite'.
+                            enum:
+                            - Overwrite
+                            - Merge
+                            type: string
                           to:
                             description: |-
                               To specifies the destination path within the artifact.

config/testdata/composition/generators.yaml

@@ -18,5 +18,9 @@ spec:
         - from: "@chart/**"
           to: "@artifact/"
           exclude: ["*.md"]
+        - from: "@repo/charts/podinfo/values.yaml"
+          to: "@artifact/podinfo/values.yaml"
+          strategy: Overwrite
         - from: "@repo/charts/podinfo/values-prod.yaml"
           to: "@artifact/podinfo/values.yaml"
+          strategy: Merge

docs/spec/v1beta1/artifactgenerators.md

@@ -96,11 +96,12 @@ spec:
           to: "@artifact/"
         - from: "@repo/charts/podinfo/values-prod.yaml"
           to: "@artifact/podinfo/values.yaml"
+          strategy: Merge # Or `Overwrite` to replace the values.yaml
 ```
 
 The above generator will create an ExternalArtifact named `podinfo-composite` in the `apps` namespace,
-which contains the Helm chart from the `podinfo-chart` OCI repository with the `values.yaml` overwritten
-by the `values-prod.yaml` file from the Git repository.
+which contains the Helm chart from the `podinfo-chart` OCI repository with the `values.yaml` merged with
+`values-prod.yaml` from the Git repository.
 
 The generated ExternalArtifact can be deployed using a Flux HelmRelease, for example:
 
@@ -263,6 +264,8 @@ Each copy operation specifies how to copy files from sources into the generated
   the root of the generated artifact and `path` is the relative path to a file or directory.
 - `exclude` (optional): A list of glob patterns to filter out from the source selection.
   Any file matched by `from` that also matches an exclude pattern will be ignored.
+- `strategy` (optional): Can be `Overwrite` (default) or `Merge`. The `Merge` strategy
+  only works for YAML files and merges using the same logic as Helm for `values.yaml` files.
 
 Copy operations use `cp`-like semantics:
 
@@ -297,6 +300,21 @@ Examples of copy operations:
     - "**/testdata/**"             # Excludes all files under any testdata/ dir
 ```
 
+Example of copy with `Merge` strategy:
+
+```yaml
+# Copy the chart contents (this includes chart-name/values.yaml)
+- from: "@chart/"
+  to: "@artifact/"
+# Merge values.yaml files - (like `helm --values values-prod.yaml`)
+- from: "@git/values-prod.yaml"
+  to: "@artifact/chart-name/values.yaml"
+  strategy: Merge
+```
+
+**Note** that the merge strategy will replace _arrays_ entirely,
+the behavior is the same as Helm's `values.yaml` merging.
+
 ## Working with ArtifactGenerators
 
 ### Suspend and Resume Reconciliation