MDBF-1109 - Auto update the Debian Sid environment

[skip ci]

Quick recap on how images are deployed to production:
1. On Pull Request : an image is only built
2. On Push to DEV branch: the image is built and pushed to quay / ghcr but with a `dev_` prefix tag. `buildbot.dev.mariadb.org` uses images starting with`dev_`
3. On push to MAIN branch: the `dev_`  tag is moved in quay / ghcr replacing the production image that doesn't have `dev_` in its name. `buildbot.mariadb.org` looks for images not having `dev_` in their name.

On schedule goal:
- on schedule event (which by GitHub laws is triggered on the default branch >>dev<<). We want to build the container image, push it to dev and then deploy it to prod.

Framework:
The dispatcher is scheduled to run on the third of the month. Child workflows having `is_scheduled_event` input will be triggered.

In the child workflow only the images with `deploy_on_schedule: true` are rebuilt.

In the workflow template `bbw_build_container_template.yml`:
-  the job runs if the run is scheduled and the matrix item has 'deploy_on_schedule: true` or for any other event type (pull request, push)

If `is_scheduled_event` is True then tags are moved to Production.

Author: RazvanLiviuVarzaru

.github/workflows/REAMDE.md

@@ -0,0 +1,80 @@
+# Building containerized build environments for Buildbot
+
+**A build environment is a Docker container image used by Buildbot to build and test the server.**
+
+Most of the build environments are created using `build-` workflows that share the same template file: [bbw_build_container_template.yml](bbw_build_container_template.yml).
+An exception is [bbw_build_container_rhel.yml](bbw_build_container_rhel.yml), which contains special tasks only applicable to `RHEL`.
+
+Each workflow defines a matrix of images to be built. Adding a new image is as simple as adding a new item to the matrix. See **Adding a new image**.
+
+## From Build to Production
+
+Workflows are triggered by `Dockerfile` changes in [/ci_build_images/](../../ci_build_images/)
+
+In **quay** and **ghcr**, we use two types of tags:
+
+1. `dev_#image_name#` — used by `buildbot.dev.mariadb.org`
+1. `#image_name#` — used by `buildbot.mariadb.org`
+
+To update a **production** image, create a Pull Request, merge it into the `DEV` branch, and then sync the changes to `MAIN`.
+
+## Events that trigger a workflow
+
+1. **Pull Request** – The image is built, and the result is reported as a CI Check.
+1. **Push to DEV** – The image is built and pushed to **quay** and **ghcr** with the tag `dev_#image_name#`.
+1. **Push to MAIN** – The `dev_#image_name#` tag is moved to `#image_name#` in both **quay** and **ghcr**.
+1. **Schedule** – See **Workflow dispatcher**.
+
+## Adding a new image
+
+You need to identify the group where the new image belongs.
+For example, adding `Fedora 42` means modifying [build-fedora-based.yml](build-fedora-based.yml).
+
+An item should be added under `matrix/include`. For example:
+
+```
+  matrix:
+    include:
+      - image: debian:11
+        platforms: linux/amd64, linux/arm64/v8
+        branch: 10.11
+        nogalera: false
+```
+
+**Required parameters**:
+
+- **Image** – This is the value used in the `FROM` instruction of the Dockerfile.
+- **Platforms** – Specify more than one to build a multi-architecture container image.
+- **Dockerfile** – A space-delimited list of `Dockerfiles` from the **ci_build_images** directory. The order is important, as they are concatenated.
+
+**Optional**:
+
+- **tag** – The displayed tag in **quay** and **ghcr**. If not specified, it's the same as the image name with `:` removed.
+- **runner** – GitHub Runner used for the build.
+- **clang_version** – Only relevant for [ci_build_images/msan.fragment.Dockerfile](msan.fragment.Dockerfile).
+- **branch** – For `debian`-based `Dockerfiles`, this installs build dependencies based on the control file from the specified MariaDB branch, e.g. `mk-build-deps -r -i debian/control`.
+- **install_valgrind** – Installs Valgrind in the final container image. Required for builders running tests under Valgrind.
+- **files** – JSON list of repository files needed in the final container image.
+- **nogalera** – If **True**, `galera-4` will not be installed in the container. Set to **True** when no Galera package is available on `ci.mariadb.org` for the distribution.
+- **deploy_on_schedule** – If **True**, the image will be rebuilt and deployed to production on the schedule defined by the **workflow dispatcher**.
+
+## Workflow dispatcher
+
+All `build-` workflows can be manually dispatched using `build-workflow-dispatcher.yml`.
+
+When dispatching the workflow, behavior depends on the `source branch`:
+
+- **DEV** – Builds the image and pushes it to **quay** and **ghcr** with the `dev_#image_name#` tag.
+- **MAIN** – Performs a production deployment by moving the `dev_#image_name#` tag to `#image_name#`.
+
+### Scheduler
+
+On the specified schedule, this workflow will trigger (on the default branch) all workflows that implement `is_scheduled_event` as an input to the `workflow_call` event.
+
+```
+on:
+  schedule:
+    - cron: '0 3 3 * *' # Third of the month
+```
+
+During this event, the image is not only rebuilt and pushed to **quay** and **ghcr**, but the tag is also moved and deployed to **production**.

.github/workflows/bbw_build_container_template.yml

@@ -39,9 +39,19 @@ on:
         required: false
         type: string
         default: 'false'
+      deploy_on_schedule:
+        required: false
+        type: boolean
+        default: false
+
+      is_scheduled_event:
+        required: false
+        type: boolean
+        default: false
 
 jobs:
   build:
+    if: ${{ (!inputs.is_scheduled_event) || (inputs.is_scheduled_event && inputs.deploy_on_schedule) }}
     runs-on: ${{ inputs.runner || 'ubuntu-22.04' }}
     services:
       registry:
@@ -198,7 +208,7 @@ jobs:
 
 
       - name: ghcr.io - move tag to production
-        if: ${{ env.DEPLOY_IMAGES == 'true' && env.MAIN_BRANCH == 'true' }}
+        if: ${{ env.DEPLOY_IMAGES == 'true' && (inputs.is_scheduled_event == 'true' || env.MAIN_BRANCH == 'true') }}
         run: |
           msg="Update tag (dev_${{ env.IMG }} --> ${{ env.IMG }})"
           line="${msg//?/=}"
@@ -226,7 +236,7 @@ jobs:
             docker://quay.io/mariadb-foundation/${{ env.REPO }}:dev_${{ env.IMG }}
 
       - name: quay.io - move tag to production
-        if: ${{ env.DEPLOY_IMAGES == 'true' && env.MAIN_BRANCH =='true' }}
+        if: ${{ env.DEPLOY_IMAGES == 'true' && (inputs.is_scheduled_event == 'true' || env.MAIN_BRANCH == 'true')}}
         run: |
           msg="Update tag (dev_${{ env.IMG }} --> ${{ env.IMG }})"
           line="${msg//?/=}"

.github/workflows/build-debian-based.yml

@@ -24,6 +24,11 @@ on:
       - .github/workflows/bbw_build_container_template.yml
 
   workflow_call:
+    inputs:
+      is_scheduled_event:
+        required: false
+        type: boolean
+        default: false
 
 jobs:
   build-images:
@@ -65,12 +70,14 @@ jobs:
             platforms: linux/amd64, linux/arm64/v8, linux/ppc64le
             branch: 11.4
             nogalera: false
+            deploy_on_schedule: true
 
           - image: debian:sid
             platforms: linux/386
             branch: 11.4
             tag: debiansid-386
             nogalera: false
+            deploy_on_schedule: true
 
           - image: ubuntu:22.04
             platforms: linux/amd64, linux/arm64/v8, linux/ppc64le, linux/s390x
@@ -95,4 +102,6 @@ jobs:
       tag: ${{ matrix.tag }}
       branch: ${{ matrix.branch }}
       nogalera: ${{ matrix.nogalera }}
+      is_scheduled_event: ${{ inputs.is_scheduled_event || false }}
+      deploy_on_schedule: ${{ matrix.deploy_on_schedule || false }}
     secrets: inherit

.github/workflows/build-workflow-dispatcher.yml

@@ -1,6 +1,9 @@
 name: Dispatch all build container images workflows
 
 on:
+  schedule:
+    - cron: '0 12 5 * *' # FIXME: Adjust after first run
+
   workflow_dispatch:
     inputs:
       build-centospip-based:
@@ -50,9 +53,11 @@ jobs:
     uses: ./.github/workflows/build-centos.pip-based.yml
     secrets: inherit
   build-debian-based:
-    if: ${{ inputs.build-debian-based }}
+    if: ${{ inputs.build-debian-based || github.event_name == 'schedule'}}
     uses: ./.github/workflows/build-debian-based.yml
     secrets: inherit
+    with:
+      is_scheduled_event: ${{ github.event_name == 'schedule' }}
   build-debian-aocc-based:
     if: ${{ inputs.build-debian-aocc-based }}
     uses: ./.github/workflows/build-debian.aocc-based.yml