review feedback comments

Author: adwk67

docs/modules/demos/pages/airflow-scheduled-job.adoc

@@ -131,31 +131,6 @@ image::airflow-scheduled-job/airflow_11.png[]
 
 == Patching Airflow to stress-test DAG parsing using relevant environment variables
 
-By default, Airflow runs database intialization routines on start-up.
-These check that an Admin user exists and that the database schema is up-to-date.
-Since they are idempotent and invoke little overhead, they can be run safely each time for most environments.
-If, however, it makes sense to deactivate this, it can be turned off by patching the running cluster with a resource definition such as this:
-
-[source,yaml]
-----
----
-apiVersion: airflow.stackable.tech/v1alpha1
-kind: AirflowCluster
-metadata:
-  name: airflow
-spec:
-  clusterConfig:
-    databaseInitialization:
-      enabled: false  # <1>
-----
-<1> Turn off the initialization routine by setting `databaseInitialization.enabled` to `false`
-
-NOTE: The field `databaseInitialization.enabled` is `true` by default to be backwards-compatible.
-A fresh Airflow cluster cannot be created with this field set to `false` as this results in missing metadata in the Airflow database.
-
-WARNING: Setting `databaseInitialization.enabled` to `false` is an unsupported operation as subsequent updates to a running Airflow cluster can result in broken behaviour due to inconsistent metadata.
-Only set `databaseInitialization.enabled` to `false` if you know what you are doing!
-
 The demo also created a third DAG in the ConfigMap, called `dag_factory.py`, which was not mounted to the cluster and therefore does not appear in the UI.
 This DAG can be used to create a number of individual DAGs on-the-fly, thus allowing a certain degree of stress-testing of the DAG scan/register steps (the generated DAGs themselves are trivial and so this approach will not really increase the burden of DAG _parsing_).
 To include this in the list of DAGs (without removing the existing ones), an extra volumeMount is needed, as shown below.
@@ -170,8 +145,6 @@ metadata:
   name: airflow
 spec:
   clusterConfig:
-    databaseInitialization:
-      enabled: false
     volumeMounts:
       - name: airflow-dags
         mountPath: /dags/dag_factory.py
@@ -203,6 +176,18 @@ spec:
         envOverrides: *envOverrides
 ----
 
+THe patch can be applied like this:
+
+[source,console]
+----
+kubectl patch airflowcluster airflow --type="merge" --patch-file stacks/airflow/patch_airflow.yaml
+----
+
+[NOTE]
+====
+The scheduled job runs every minute and so an instance of it may be running while the scheduler is being re-started as a result of the patch, in which case that instance may fail.
+====
+
 == Summary
 
 This demo showed how DAGs can be made available for Airflow, scheduled, run and then inspected with the Webserver UI.

stacks/_templates/redis-airflow.yaml

@@ -345,4 +345,3 @@ stringData:
   adminUser.password: "{{ airflowAdminPassword }}"
   connections.secretKey: "{{ airflowSecretKey }}"
   connections.sqlalchemyDatabaseUri: postgresql+psycopg2://airflow:airflow@postgresql-airflow/airflow
-  connections.celeryResultBackend: db+postgresql://airflow:airflow@postgresql-airflow/airflow

stacks/airflow/airflow.yaml

@@ -5,8 +5,6 @@ metadata:
   name: airflow
 spec:
   clusterConfig:
-    databaseInitialization:
-      enabled: false
     volumeMounts:
       - name: airflow-dags
         mountPath: /dags/dag_factory.py