You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
WARNING: This demo should not be run alongside other demos.
71
+
### 2. GitOps - interact with a Git repository to apply changes made in code into the cluster
72
+
73
+
In addition to the exploration part, users can interact with a forked repository and apply changes to the Git repository.
74
+
These changes then are automatically synced into the cluster.
75
+
76
+
In order to interact with the Git repository, the {stackable-demo-repository}[stackable demo repository] must be forked and additional parameters must be provided to `stackablectl`.
77
+
This is explained in more detail in the section <<interact-with-git-repository,How to interact with the Git repository>> at the bottom.
NOTE: ArgoCD will be deployed in the `argo-cd` namespace by `stackablectl`.
71
85
ArgoCD itself will create other namespaces for the deployed products.
@@ -126,7 +140,8 @@ using different versions and Git sources (repository & branch) as well as the po
126
140
127
141
NOTE: This demo does not use a multi cluster environment for the sake of simplicity.
128
142
129
-
The following part dives deeper into the definition of the Stackable operator `ApplicationSet` and can be skipped.
143
+
The following part dives deeper into the definition of the Stackable operator `ApplicationSet` consumed by ArgoCD.
144
+
Users not interested in the technical parts of the `ApplicationSet` might skip this section.
130
145
131
146
.Stackable operators `ApplicationSet` details
132
147
[%collapsible]
@@ -191,7 +206,7 @@ spec:
191
206
<1> List of Stackable operators to install.
192
207
<2> List of clusters and Stackable release versions for each cluster.
193
208
194
-
The `matrix.generators.list[].elements[]` will create a union of parameters that may be used in the `ApplicationSet` template as follows:
209
+
The `matrix.generators.list[].elements[]` will create a union of parameters (e.g. `operator`, `cluster`, `server` and `targetRevision`), that may be used in the `ApplicationSet` template as follows:
* Experiment with DataOps (e.g., Airflow and Trino).
322
-
323
313
[#interact-with-git-repository]
324
314
== How to interact with ArgoCD, Airflow and the Git repository
325
315
326
-
Since this Demo is hosted in the {stackable-demo-repository}[Stackable Demo repository], where merging etc. requires approval, the recommendation is to fork the {stackable-demo-repository}[Stackable Demo repository].
316
+
Since this Demo is hosted in the {stackable-demo-repository}[Stackable Demo repository], where merging etc. requires approval,
317
+
the recommendation is to fork and clone the {stackable-demo-repository}[Stackable Demo repository].
318
+
This requires a two-step process:
327
319
328
-
=== Forking the demo repository
320
+
=== 1. Forking the demo repository
329
321
330
322
This {github-fork}[GitHub tutorial] shows how to fork a repository.
After forking the demo repository, a local copy can be cloned and the `stackablectl` install command must be parameterized with the fork URL and branch.
334
+
After forking and cloning a local copy, the `stackablectl` install command must be parameterized with the forked repository URL and branch.
Edit manifests or add new DAG files within your cloned repository:
343
+
After obaining a local copy, changes made to Stackable manifests or Airflow DAGs can be integrated via GitOps.
344
+
Edit Stackable manifests or add new DAGs within your cloned repository:
352
345
353
346
* Manifests are in: `demos/argo-cd-git-ops/manifests/`
354
347
* Airflow DAGs are in: `demos/argo-cd-git-ops/dags/`
355
348
356
-
==== Increase Airflow webserver replicas
349
+
=== 1. Increase Airflow webserver replicas
357
350
358
351
Assuming your working directory is the root of the forked demo repository, try to increase the `spec.webservers.roleGroups.<role-group>.replicas` in the folder `demos/argo-cd-git-ops/manifests/airflow/airflow.yaml`.
359
352
Once this is pushed / merged, ArgoCD should sync the changes and you should see more webserver pods.
360
353
361
-
==== Add new Airflow DAGs
354
+
=== 2. Add new Airflow DAGs
362
355
363
356
In the `demos/argo-cd-git-ops/manifests/airflow/airflow.yaml` manifest you have to adapt the git-sync configuration for DAGs to the forked repository:
364
357
@@ -398,7 +391,7 @@ File Path PID Run
398
391
399
392
If another DAG is displayed, try to refresh the Airflow UI if changes have not been propagated yet.
0 commit comments