Skip to content

Commit e6c7573

Browse files
committed
updated docs
1 parent 4b26f6b commit e6c7573

File tree

2 files changed

+12
-6
lines changed

2 files changed

+12
-6
lines changed

docs/modules/demos/pages/airflow-scheduled-job.adoc

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -129,7 +129,7 @@ asynchronously - and another to poll the running job to report on its status.
129129

130130
image::airflow-scheduled-job/airflow_11.png[]
131131

132-
== Patching Airflow to deactivate database initialization
132+
== Patching Airflow to stress-test DAG parsing using relevant environment variables
133133

134134
By default, Airflow runs database intialization routines on start-up.
135135
These check that an Admin user exists and that the database schema is up-to-date.
@@ -145,13 +145,17 @@ metadata:
145145
name: airflow
146146
spec:
147147
clusterConfig:
148-
dbInit: false # <1>
148+
databaseInitialization:
149+
enabled: false # <1>
149150
----
150-
<1> Turn off the initialization routine by setting `dbInit` to `false`
151+
<1> Turn off the initialization routine by setting `databaseInitialization.enabled` to `false`
151152

152-
NOTE: The field `dbInit` is `true` by default to be backwards-compatible.
153+
NOTE: The field `databaseInitialization.enabled` is `true` by default to be backwards-compatible.
153154
A fresh Airflow cluster cannot be created with this field set to `false` as this results in missing metadata in the Airflow database.
154155

156+
WARNING: Setting `databaseInitialization.enabled` to `false` is an unsupported operation as subsequent updates to a running Airflow cluster can result in broken behaviour due to inconsistent metadata.
157+
Only set `databaseInitialization.enabled` to `false` if you know what you are doing!
158+
155159
The demo also created a third DAG in the ConfigMap, called `dag_factory.py`, which was not mounted to the cluster and therefore does not appear in the UI.
156160
This DAG can be used to create a number of individual DAGs on-the-fly, thus allowing a certain degree of stress-testing of the DAG scan/register steps (the generated DAGs themselves are trivial and so this approach will not really increase the burden of DAG _parsing_).
157161
To include this is in the list of DAGs (without removing the existing ones), an extra volumeMount is needed, as shown below.
@@ -166,7 +170,8 @@ metadata:
166170
name: airflow
167171
spec:
168172
clusterConfig:
169-
dbInit: false
173+
databaseInitialization:
174+
enabled: false
170175
volumeMounts:
171176
- name: airflow-dags
172177
mountPath: /dags/dag_factory.py

stacks/airflow/patch_airflow.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,8 @@ metadata:
55
name: airflow
66
spec:
77
clusterConfig:
8-
dbInit: false
8+
databaseInitialization:
9+
enabled: false
910
volumeMounts:
1011
- name: airflow-dags
1112
mountPath: /dags/dag_factory.py

0 commit comments

Comments
 (0)