Skip to content

Commit 47e2d63

Browse files
authored
Argocd: Work in review feedback for ArgoCD demo (#296)
* review feedback fabian * switch conclusion & how to interact with git
1 parent 3e9a47e commit 47e2d63

File tree

1 file changed

+59
-41
lines changed

1 file changed

+59
-41
lines changed

docs/modules/demos/pages/argo-cd-git-ops.adoc

Lines changed: 59 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ You will learn how to:
2626
* Update Airflow DAGs or modify deployments simply by committing to Git.
2727

2828
This hands-on approach illustrates how GitOps can simplify application lifecycle management and enforce a clear,
29-
auditable workflow across environments (development, staging, production).
29+
auditable workflow across single or multiple Kubernetes environments (development, staging, production).
3030

3131
All products and manifests are synced and deployed via ArgoCD (except ArgoCD itself, which is bootstrapped via `stackablectl`).
3232

@@ -55,17 +55,31 @@ image::argo-cd-git-ops/architecture-overview.drawio.svg[]
5555

5656
## Installation
5757

58-
Install this demo on an existing Kubernetes cluster:
58+
WARNING: This demo should not be run alongside other demos.
59+
60+
This demo can be installed on an existing Kubernetes cluster in two ways:
5961

60-
NOTE: In order to interact with the Git repository, this repository must be forked and additional parameters must be provided to `stackablectl`.
61-
This is explained in more detail in the section <<interact-with-git-repository,How to interact with the Git repository>>.
62+
### 1. Exploration - interact with the ArgoCD, Airflow and Minio web UI
63+
64+
The "easy" way. Checkout the ArgoCD, Airflow and Minio web UI, start DAGs and explore logs.
6265

6366
[source,console]
6467
----
6568
$ stackablectl demo install argo-cd-git-ops --namespace argo-cd
6669
----
6770

68-
WARNING: This demo should not be run alongside other demos.
71+
### 2. GitOps - interact with a Git repository to apply changes made in code into the cluster
72+
73+
In addition to the exploration part, users can interact with a forked repository and apply changes to the Git repository.
74+
These changes then are automatically synced into the cluster.
75+
76+
In order to interact with the Git repository, the {stackable-demo-repository}[stackable demo repository] must be forked and additional parameters must be provided to `stackablectl`.
77+
This is explained in more detail in the section <<interact-with-git-repository,How to interact with the Git repository>> at the bottom.
78+
79+
[source,console]
80+
----
81+
stackablectl demo install argo-cd-git-ops --namespace argo-cd --parameters customGitUrl=<my-demo-fork-url> --parameters customGitBranch=<my-custom-branch-with-changes>
82+
----
6983

7084
NOTE: ArgoCD will be deployed in the `argo-cd` namespace by `stackablectl`.
7185
ArgoCD itself will create other namespaces for the deployed products.
@@ -126,7 +140,8 @@ using different versions and Git sources (repository & branch) as well as the po
126140

127141
NOTE: This demo does not use a multi cluster environment for the sake of simplicity.
128142

129-
The following part dives deeper into the definition of the Stackable operator `ApplicationSet` and can be skipped.
143+
The following part dives deeper into the definition of the Stackable operator `ApplicationSet` consumed by ArgoCD.
144+
Users not interested in the technical parts of the `ApplicationSet` might skip this section.
130145

131146
.Stackable operators `ApplicationSet` details
132147
[%collapsible]
@@ -191,7 +206,7 @@ spec:
191206
<1> List of Stackable operators to install.
192207
<2> List of clusters and Stackable release versions for each cluster.
193208
194-
The `matrix.generators.list[].elements[]` will create a union of parameters that may be used in the `ApplicationSet` template as follows:
209+
The `matrix.generators.list[].elements[]` will create a union of parameters (e.g. `operator`, `cluster`, `server` and `targetRevision`), that may be used in the `ApplicationSet` template as follows:
195210
196211
[source,yaml]
197212
----
@@ -295,41 +310,18 @@ image::argo-cd-git-ops/minio-dag-run-logs.png[]
295310

296311
The log files contained in the single folders are the same as the logs shown above in the Airflow web UI.
297312

298-
== Conclusion
299-
300-
This demo acts as a blueprint for showing how complex data platform components can be managed with ArgoCD and GitOps.
301-
Once familiar with this pattern, you can extend it to multi-cluster environments, add CI/CD pipelines for automated manifest testing,
302-
or integrate external secret stores like HashiCorp Vault for production use.
303-
This setup lays the foundation for a fully automated, scalable, and secure Kubernetes-based data platform.
304-
305-
This tutorial demonstrates how ArgoCD and Stackable can be combined to deliver a streamlined GitOps experience:
306-
307-
* All cluster resources and workloads are managed declaratively via Git.
308-
* ArgoCD continuously ensures the cluster state matches Git.
309-
* Sealed Secrets provide secure and auditable secret management.
310-
* Airflow DAG updates occur automatically by committing code to the repository.
311-
312-
This approach scales naturally across environments - development, staging, and production - while reducing manual operations, improving visibility,
313-
and enforcing consistency.
314-
By adopting GitOps with ArgoCD and Stackable, teams gain a clear, auditable, and automated path from code to production.
315-
316-
Next steps:
317-
318-
* Explore multi-cluster ApplicationSet deployments to target multiple Kubernetes clusters.
319-
* Integrate CI workflows to automatically validate and merge manifest updates.
320-
* Expand beyond Airflow to manage additional Stackable components (e.g., Kafka, Trino, Superset).
321-
* Experiment with DataOps (e.g., Airflow and Trino).
322-
323313
[#interact-with-git-repository]
324314
== How to interact with ArgoCD, Airflow and the Git repository
325315

326-
Since this Demo is hosted in the {stackable-demo-repository}[Stackable Demo repository], where merging etc. requires approval, the recommendation is to fork the {stackable-demo-repository}[Stackable Demo repository].
316+
Since this Demo is hosted in the {stackable-demo-repository}[Stackable Demo repository], where merging etc. requires approval,
317+
the recommendation is to fork and clone the {stackable-demo-repository}[Stackable Demo repository].
318+
This requires a two-step process:
327319

328-
=== Forking the demo repository
320+
=== 1. Forking the demo repository
329321

330322
This {github-fork}[GitHub tutorial] shows how to fork a repository.
331323

332-
=== Cloning the demo repository
324+
=== 2. Cloning the demo repository
333325

334326
Clone the demo repository:
335327

@@ -339,26 +331,27 @@ git clone https://github.com/<your-username>/demos.git
339331
cd demos
340332
----
341333

342-
After forking the demo repository, a local copy can be cloned and the `stackablectl` install command must be parameterized with the fork URL and branch.
334+
After forking and cloning a local copy, the `stackablectl` install command must be parameterized with the forked repository URL and branch.
343335

344336
[source,console]
345337
----
346338
stackablectl demo install argo-cd-git-ops --namespace argo-cd --parameters customGitUrl=<my-demo-fork-url> --parameters customGitBranch=<my-custom-branch-with-changes>
347339
----
348340

349-
=== Making changes to the repository
341+
== Making changes to the repository
350342

351-
Edit manifests or add new DAG files within your cloned repository:
343+
After obaining a local copy, changes made to Stackable manifests or Airflow DAGs can be integrated via GitOps.
344+
Edit Stackable manifests or add new DAGs within your cloned repository:
352345

353346
* Manifests are in: `demos/argo-cd-git-ops/manifests/`
354347
* Airflow DAGs are in: `demos/argo-cd-git-ops/dags/`
355348

356-
==== Increase Airflow webserver replicas
349+
=== 1. Increase Airflow webserver replicas
357350

358351
Assuming your working directory is the root of the forked demo repository, try to increase the `spec.webservers.roleGroups.<role-group>.replicas` in the folder `demos/argo-cd-git-ops/manifests/airflow/airflow.yaml`.
359352
Once this is pushed / merged, ArgoCD should sync the changes and you should see more webserver pods.
360353

361-
==== Add new Airflow DAGs
354+
=== 2. Add new Airflow DAGs
362355

363356
In the `demos/argo-cd-git-ops/manifests/airflow/airflow.yaml` manifest you have to adapt the git-sync configuration for DAGs to the forked repository:
364357

@@ -398,7 +391,7 @@ File Path PID Run
398391

399392
If another DAG is displayed, try to refresh the Airflow UI if changes have not been propagated yet.
400393

401-
==== Commit and push changes
394+
=== 3. Commit and push changes
402395

403396
[source,console]
404397
----
@@ -409,3 +402,28 @@ git push origin <my-custom-branch-with-changes>
409402
----
410403

411404
Now ArgoCD and Airflow should sync the respective changes into the cluster.
405+
406+
== Conclusion
407+
408+
This demo acts as a blueprint for showing how complex data platform components can be managed with ArgoCD and GitOps.
409+
Once familiar with this pattern, you can extend it to multi-cluster environments, add CI/CD pipelines for automated manifest testing,
410+
or integrate external secret stores like HashiCorp Vault for production use.
411+
This setup lays the foundation for a fully automated, scalable, and secure Kubernetes-based data platform.
412+
413+
This tutorial demonstrates how ArgoCD and Stackable can be combined to deliver a streamlined GitOps experience:
414+
415+
* All cluster resources and workloads are managed declaratively via Git.
416+
* ArgoCD continuously ensures the cluster state matches Git.
417+
* Sealed Secrets provide secure and auditable secret management.
418+
* Airflow DAG updates occur automatically by committing code to the repository.
419+
420+
This approach scales naturally across environments - development, staging, and production - while reducing manual operations, improving visibility,
421+
and enforcing consistency.
422+
By adopting GitOps with ArgoCD and Stackable, teams gain a clear, auditable, and automated path from code to production.
423+
424+
Next steps:
425+
426+
* Explore multi-cluster ApplicationSet deployments to target multiple Kubernetes clusters.
427+
* Integrate CI workflows to automatically validate and merge manifest updates.
428+
* Expand beyond Airflow to manage additional Stackable components (e.g., Kafka, Trino, Superset).
429+
* Experiment with DataOps (e.g., Airflow and Trino).

0 commit comments

Comments
 (0)