|
| 1 | +# Building containerized build environments for Buildbot |
| 2 | + |
| 3 | +**A build environment is a Docker container image used by Buildbot to build and test the server.** |
| 4 | + |
| 5 | +Most of the build environments are created using `build-` workflows that share the same template file: [bbw_build_container_template.yml](bbw_build_container_template.yml). |
| 6 | +An exception is [bbw_build_container_rhel.yml](bbw_build_container_rhel.yml), which contains special tasks only applicable to `RHEL`. |
| 7 | + |
| 8 | +Each workflow defines a matrix of images to be built. Adding a new image is as simple as adding a new item to the matrix. See **Adding a new image**. |
| 9 | + |
| 10 | +## From Build to Production |
| 11 | + |
| 12 | +Workflows are triggered by `Dockerfile` changes in [/ci_build_images/](../../ci_build_images/) |
| 13 | + |
| 14 | +In **quay** and **ghcr**, we use two types of tags: |
| 15 | +1. `dev_#image_name#` — used by `buildbot.dev.mariadb.org` |
| 16 | +2. `#image_name#` — used by `buildbot.mariadb.org` |
| 17 | + |
| 18 | +To update a **production** image, create a Pull Request, merge it into the `DEV` branch, and then sync the changes to `MAIN`. |
| 19 | + |
| 20 | +## Events that trigger a workflow |
| 21 | + |
| 22 | +1. **Pull Request** – The image is built, and the result is reported as a CI Check. |
| 23 | +2. **Push to DEV** – The image is built and pushed to **quay** and **ghcr** with the tag `dev_#image_name#`. |
| 24 | +3. **Push to MAIN** – The `dev_#image_name#` tag is moved to `#image_name#` in both **quay** and **ghcr**. |
| 25 | +4. **Schedule** – See **Workflow dispatcher**. |
| 26 | + |
| 27 | +## Adding a new image |
| 28 | + |
| 29 | +You need to identify the group where the new image belongs. |
| 30 | +For example, adding `Fedora 42` means modifying [build-fedora-based.yml](build-fedora-based.yml). |
| 31 | + |
| 32 | +An item should be added under `matrix/include`. For example: |
| 33 | + |
| 34 | +``` |
| 35 | + matrix: |
| 36 | + include: |
| 37 | + - image: debian:11 |
| 38 | + platforms: linux/amd64, linux/arm64/v8 |
| 39 | + branch: 10.11 |
| 40 | + nogalera: false |
| 41 | +``` |
| 42 | + |
| 43 | +**Required parameters**: |
| 44 | +- **Image** – This is the value used in the `FROM` instruction of the Dockerfile. |
| 45 | +- **Platforms** – Specify more than one to build a multi-architecture container image. |
| 46 | +- **Dockerfile** – A space-delimited list of `Dockerfiles` from the **ci_build_images** directory. The order is important, as they are concatenated. |
| 47 | + |
| 48 | +**Optional**: |
| 49 | +- **tag** – The displayed tag in **quay** and **ghcr**. If not specified, it's the same as the image name with `:` removed. |
| 50 | +- **runner** – GitHub Runner used for the build. |
| 51 | +- **clang_version** – Only relevant for [ci_build_images/msan.fragment.Dockerfile](msan.fragment.Dockerfile). |
| 52 | +- **branch** – For `debian`-based `Dockerfiles`, this installs build dependencies based on the control file from the specified MariaDB branch, e.g. `mk-build-deps -r -i debian/control`. |
| 53 | +- **install_valgrind** – Installs Valgrind in the final container image. Required for builders running tests under Valgrind. |
| 54 | +- **files** – JSON list of repository files needed in the final container image. |
| 55 | +- **nogalera** – If **True**, `galera-4` will not be installed in the container. Set to **True** when no Galera package is available on `ci.mariadb.org` for the distribution. |
| 56 | +- **deploy_on_schedule** – If **True**, the image will be rebuilt and deployed to production on the schedule defined by the **workflow dispatcher**. |
| 57 | + |
| 58 | +## Workflow dispatcher |
| 59 | + |
| 60 | +All `build-` workflows can be manually dispatched using `build-workflow-dispatcher.yml`. |
| 61 | + |
| 62 | +When dispatching the workflow, behavior depends on the `source branch`: |
| 63 | +- **DEV** – Builds the image and pushes it to **quay** and **ghcr** with the `dev_#image_name#` tag. |
| 64 | +- **MAIN** – Performs a production deployment by moving the `dev_#image_name#` tag to `#image_name#`. |
| 65 | + |
| 66 | +### Scheduler |
| 67 | + |
| 68 | +On the specified schedule, this workflow will trigger (on the default branch) all workflows that implement `is_scheduled_event` as an input to the `workflow_call` event. |
| 69 | + |
| 70 | +``` |
| 71 | +on: |
| 72 | + schedule: |
| 73 | + - cron: '0 3 3 * *' # Third of the month |
| 74 | +``` |
| 75 | + |
| 76 | +During this event, the image is not only rebuilt and pushed to **quay** and **ghcr**, but the tag is also moved and deployed to **production**. |
0 commit comments