|
| 1 | +# CI Builds via GNU make and GitHub Actions |
| 2 | + |
| 3 | +## Contents |
| 4 | + |
| 5 | +<!-- mdformat-toc start --slug=github --no-anchors --maxlevel=6 --minlevel=2 --> |
| 6 | + |
| 7 | +- [Contents](#contents) |
| 8 | +- [Make-based actions & workflows in `core-crypto`](#make-based-actions--workflows-in-core-crypto) |
| 9 | +- [Generic `make` action](#generic-make-action) |
| 10 | + - [File Hashes of Dependencies](#file-hashes-of-dependencies) |
| 11 | + - [Artifact Caching](#artifact-caching) |
| 12 | +- [Specialized `make` actions](#specialized-make-actions) |
| 13 | + |
| 14 | +<!-- mdformat-toc end --> |
| 15 | + |
| 16 | +## Make-based actions & workflows in `core-crypto` |
| 17 | + |
| 18 | +Our CI pipeline is heavily driven by make rules (in the root [`Makefile`](../Makefile)). To avoid duplicating logic, the |
| 19 | +CI uses small GitHub composite actions that invoke specific make targets with consistent artifact caching via GitHub |
| 20 | +artifatcs. |
| 21 | + |
| 22 | +The directory `.github/actions/make/` houses: |
| 23 | + |
| 24 | +- A generic composite action ([`.github/actions/make/action.yml`](../.github/actions/make/action.yml)) |
| 25 | +- Specialized wrappers in subdirectories which call the generic action |
| 26 | + |
| 27 | +The workflow file [`.github/workflows/pipeline.yml`](../.github/workflows/pipeline.yml) orchestrates CI jobs, and uses |
| 28 | +those make actions in steps. |
| 29 | + |
| 30 | +## Generic `make` action |
| 31 | + |
| 32 | +This action produces an artifact which corresponds to a rule in the root [`Makefile`](../Makefile), or downloads it if |
| 33 | +any subsequent workflow run already produced the artifact with unchanged prerequisites. |
| 34 | + |
| 35 | +### File Hashes of Dependencies |
| 36 | + |
| 37 | +The prerequisites of ceach artifact are defined in the [`Makefile`](../Makefile) itself. Each artifact that is produced |
| 38 | +using this action, has a corresponding variable in the [`Makefile`](../Makefile), called `<target-name>-deps`, which |
| 39 | +lists the artifact's dependencies. |
| 40 | + |
| 41 | +The [`Makefile`](../Makefile) contains a wildcard rule that uses this variable to calculate an aggregated `sha256sum` of |
| 42 | +all dependencies. This is then appended to the artifact key, identifying an artifact when downloading or uploading. |
| 43 | + |
| 44 | +### Artifact Caching |
| 45 | + |
| 46 | +Each caller of the generic `make` action must provide an artifact key, required to identify an artifact. This is |
| 47 | +necessary downloading a previously produced artifact or uploading the artifact just produced. The aggregated `sha256sum` |
| 48 | +(see above) is then appended to that key to ensure artifacts with different dependencies don't share a key. |
| 49 | + |
| 50 | +In case an artifact was successfully downloaded, the action `touch`es them, because otherwise the checked out source |
| 51 | +files would be newer than the downloaded artifacts. This is necessary because another subsequent call of this actions |
| 52 | +for another rule might require that the downloaded artifact is newer than the source files. |
| 53 | + |
| 54 | +Otherwise, the artifact is produced by calling `make <rule argument>`. |
| 55 | + |
| 56 | +In both cases (artifact was downloaded or produced), the artifact is uploaded only if it hasn't been uploaded for this |
| 57 | +workflow run of [`pipeline.yml`](../.github/workflows/pipeline.yml). This is relevant because the `make` action may be |
| 58 | +called with the same parameters multiple times in a single workflow run. |
| 59 | + |
| 60 | +## Specialized `make` actions |
| 61 | + |
| 62 | +Since most artifacts (except those produced by a job representing a "leaf" in the make dependency tree) are reused |
| 63 | +during the [`pipeline.yml`](../.github/workflows/pipeline.yml) workflow, the generic action is frequently called with |
| 64 | +the same arguments. The parameters of the `make` action are the artifact key, the `make` rule, and the target path. To |
| 65 | +avoid having to repeat the arguments when an artifact is reused, we're using sepcialized `make` actions, located in |
| 66 | +subfolders of `.github/actions/make`. These specialized actions are parameterless, except for the github token, which |
| 67 | +they just forward. The github token is needed to use the `gh` CLI on the runner machine. |
| 68 | + |
| 69 | +Any reused artifact should have such a specialized action. Whenever a new artifact is introduced in the workflow and it |
| 70 | +needs to be reused, an action should be added. |
0 commit comments