Skip to content

Commit 70c82dc

Browse files
committed
Add overview of ArtifactGenerator to readme
Signed-off-by: Stefan Prodan <[email protected]>
1 parent 785f858 commit 70c82dc

File tree

1 file changed

+169
-4
lines changed

1 file changed

+169
-4
lines changed

README.md

Lines changed: 169 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,175 @@
22

33
[![test](https://github.com/fluxcd/source-watcher/workflows/test/badge.svg)](https://github.com/fluxcd/source-watcher/actions)
44

5-
Example consumer of the GitOps Toolkit Source APIs.
5+
A Flux extension controller that enables advanced source composition and decomposition patterns for GitOps workflows.
66

7-
![Source Controller Overview](https://raw.githubusercontent.com/fluxcd/website/main/static/img/source-controller.png)
7+
## Overview
88

9-
## Guides
9+
source-watcher introduces the **ArtifactGenerator** API, which allows you to:
1010

11-
* [Watching for source changes](https://fluxcd.io/flux/gitops-toolkit/source-watcher/)
11+
- 🔗 **Compose** multiple Flux sources (GitRepository, OCIRepository, Bucket) into a single deployable artifact
12+
- 📦 **Decompose** monorepos into multiple independent artifacts with separate deployment lifecycles
13+
- 🎯 **Optimize** reconciliation by only triggering updates when specific paths change
14+
- 🏗️ **Structure** complex deployments from distributed sources maintained by different teams
15+
16+
## Quick Start
17+
18+
### Prerequisites
19+
20+
- Flux v2.7.0 or later
21+
- source-watcher v2.0.0 or later
22+
- kustomize-controller & helm-controller with `--feature-gates=ExternalArtifacts`
23+
24+
### Example: Composing Multiple Sources
25+
26+
Deploy an application that combines Kubernetes manifests from different sources:
27+
28+
```yaml
29+
apiVersion: source.extensions.fluxcd.io/v1beta1
30+
kind: ArtifactGenerator
31+
metadata:
32+
name: my-app
33+
namespace: flux-system
34+
spec:
35+
sources:
36+
- alias: backend
37+
kind: GitRepository
38+
name: backend-repo
39+
- alias: frontend
40+
kind: OCIRepository
41+
name: frontend-oci
42+
- alias: config
43+
kind: Bucket
44+
name: config-bucket
45+
artifacts:
46+
- name: my-app-composite
47+
copy:
48+
- from: "@backend/deploy/k8s/**"
49+
to: "@artifact/backend/"
50+
- from: "@frontend/k8s/manifests/*.yaml"
51+
to: "@artifact/frontend/"
52+
- from: "@config/prod/settings.yaml"
53+
to: "@artifact/config.yaml"
54+
---
55+
apiVersion: kustomize.toolkit.fluxcd.io/v1
56+
kind: Kustomization
57+
metadata:
58+
name: my-app
59+
namespace: flux-system
60+
spec:
61+
interval: 10m
62+
sourceRef:
63+
kind: ExternalArtifact
64+
name: my-app-composite
65+
path: "./"
66+
prune: true
67+
```
68+
69+
### Example: Monorepo Decomposition
70+
71+
Split a monorepo into independently deployable services:
72+
73+
```yaml
74+
apiVersion: source.extensions.fluxcd.io/v1beta1
75+
kind: ArtifactGenerator
76+
metadata:
77+
name: platform-services
78+
namespace: flux-system
79+
spec:
80+
sources:
81+
- alias: monorepo
82+
kind: GitRepository
83+
name: platform-monorepo
84+
artifacts:
85+
- name: policy-config
86+
copy:
87+
- from: "@monorepo/platform/policy/**"
88+
to: "@artifact/"
89+
- name: auth-service
90+
copy:
91+
- from: "@monorepo/services/auth/deploy/**"
92+
to: "@artifact/"
93+
- name: api-gateway
94+
copy:
95+
- from: "@monorepo/services/gateway/deploy/**"
96+
to: "@artifact/"
97+
exclude: ["**/charts/**"]
98+
```
99+
100+
Each service gets its own ExternalArtifact with an independent revision.
101+
Changes to `services/auth/` only trigger reconciliation of the auth-service,
102+
leaving other services untouched.
103+
104+
### Example: Helm Chart Composition
105+
106+
Compose Helm charts with custom values from different sources:
107+
108+
```yaml
109+
apiVersion: source.extensions.fluxcd.io/v1beta1
110+
kind: ArtifactGenerator
111+
metadata:
112+
name: podinfo
113+
namespace: apps
114+
spec:
115+
sources:
116+
- alias: chart
117+
kind: OCIRepository
118+
name: podinfo-chart
119+
namespace: apps
120+
- alias: repo
121+
kind: GitRepository
122+
name: podinfo-values
123+
namespace: apps
124+
artifacts:
125+
- name: podinfo-composite
126+
originRevision: "@chart" # Track chart version
127+
copy:
128+
- from: "@chart/"
129+
to: "@artifact/"
130+
- from: "@repo/charts/podinfo/values-prod.yaml"
131+
to: "@artifact/podinfo/values.yaml" # Override default values
132+
```
133+
134+
## Key Features
135+
136+
### Content-based Revision Tracking
137+
138+
The generated artifacts are versioned as `latest@sha256:<hash>` where the hash
139+
is computed from the artifact's content. This means:
140+
141+
- ✅ Identical content = same revision (no unnecessary reconciliations)
142+
- ✅ Any change = new revision (guaranteed updates)
143+
- ✅ Automatic change detection across all source types
144+
145+
The controller attaches the origin source revision (e.g. Git commit SHA)
146+
to the generated artifact metadata for traceability and Flux commit status updates.
147+
148+
### Flexible Copy Operations
149+
150+
Copy operations support glob patterns and exclusions:
151+
152+
- `@source/file.yaml` → `@artifact/dest/` - Copy file into directory
153+
- `@source/dir/` → `@artifact/dest/` - Copy directory as subdirectory
154+
- `@source/dir/**` → `@artifact/dest/` - Copy directory contents
155+
- `@source/file.yaml` → `@artifact/existing.yaml` - Later copy overwrites earlier ones
156+
157+
## Use Cases
158+
159+
- **Multi-Team Collaboration** - Different teams maintain different Kubernetes
160+
configurations in separate repositories. ArtifactGenerator combines them into deployable
161+
units while maintaining clear ownership boundaries.
162+
- **Environment-Specific Composition** Compose base manifests with environment-specific
163+
configurations from different sources without duplicating files across repositories.
164+
- **Vendor Chart Customization** - Combine upstream Helm charts from OCI registries with
165+
your organization's custom values and configuration overrides stored in Git.
166+
- **Selective Deployment** - Deploy only changed components from large repositories
167+
by decomposing them into focused artifacts.
168+
169+
## API Reference
170+
171+
- [ArtifactGenerator CRD](docs/spec/v1beta1/artifactgenerators.md)
172+
173+
## Contributing
174+
175+
This project is Apache 2.0 licensed and accepts contributions via GitHub pull requests.
176+
To start contributing please see the [development guide](CONTRIBUTING.md).

0 commit comments

Comments
 (0)