Skip to content

Commit 6294789

Browse files
docs: document CI workflow
1 parent c4d5874 commit 6294789

File tree

2 files changed

+71
-0
lines changed

2 files changed

+71
-0
lines changed

docs/CI.md

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
# CI Builds via GNU make and GitHub Actions
2+
3+
## Contents
4+
5+
<!-- mdformat-toc start --slug=github --no-anchors --maxlevel=6 --minlevel=2 -->
6+
7+
- [Contents](#contents)
8+
- [Make-based actions & workflows in `core-crypto`](#make-based-actions--workflows-in-core-crypto)
9+
- [Generic `make` action](#generic-make-action)
10+
- [File Hashes of Dependencies](#file-hashes-of-dependencies)
11+
- [Artifact Caching](#artifact-caching)
12+
- [Specialized `make` actions](#specialized-make-actions)
13+
14+
<!-- mdformat-toc end -->
15+
16+
## Make-based actions & workflows in `core-crypto`
17+
18+
Our CI pipeline is heavily driven by make rules (in the root [`Makefile`](../Makefile)). To avoid duplicating logic, the
19+
CI uses small GitHub composite actions that invoke specific make targets with consistent artifact caching via GitHub
20+
artifatcs.
21+
22+
The directory `.github/actions/make/` houses:
23+
24+
- A generic composite action ([`.github/actions/make/action.yml`](../.github/actions/make/action.yml))
25+
- Specialized wrappers in subdirectories which call the generic action
26+
27+
The workflow file [`.github/workflows/pipeline.yml`](../.github/workflows/pipeline.yml) orchestrates CI jobs, and uses
28+
those make actions in steps.
29+
30+
## Generic `make` action
31+
32+
This action produces an artifact which corresponds to a rule in the root [`Makefile`](../Makefile), or downloads it if
33+
any subsequent workflow run already produced the artifact with unchanged prerequisites.
34+
35+
### File Hashes of Dependencies
36+
37+
The prerequisites of ceach artifact are defined in the [`Makefile`](../Makefile) itself. Each artifact that is produced
38+
using this action, has a corresponding variable in the [`Makefile`](../Makefile), called `<target-name>-deps`, which
39+
lists the artifact's dependencies.
40+
41+
The [`Makefile`](../Makefile) contains a wildcard rule that uses this variable to calculate an aggregated `sha256sum` of
42+
all dependencies. This is then appended to the artifact key, identifying an artifact when downloading or uploading.
43+
44+
### Artifact Caching
45+
46+
Each caller of the generic `make` action must provide an artifact key, required to identify an artifact. This is
47+
necessary downloading a previously produced artifact or uploading the artifact just produced. The aggregated `sha256sum`
48+
(see above) is then appended to that key to ensure artifacts with different dependencies don't share a key.
49+
50+
In case an artifact was successfully downloaded, the action `touch`es them, because otherwise the checked out source
51+
files would be newer than the downloaded artifacts. This is necessary because another subsequent call of this actions
52+
for another rule might require that the downloaded artifact is newer than the source files.
53+
54+
Otherwise, the artifact is produced by calling `make <rule argument>`.
55+
56+
In both cases (artifact was downloaded or produced), the artifact is uploaded only if it hasn't been uploaded for this
57+
workflow run of [`pipeline.yml`](../.github/workflows/pipeline.yml). This is relevant because the `make` action may be
58+
called with the same parameters multiple times in a single workflow run.
59+
60+
## Specialized `make` actions
61+
62+
Since most artifacts (except those produced by a job representing a "leaf" in the make dependency tree) are reused
63+
during the [`pipeline.yml`](../.github/workflows/pipeline.yml) workflow, the generic action is frequently called with
64+
the same arguments. The parameters of the `make` action are the artifact key, the `make` rule, and the target path. To
65+
avoid having to repeat the arguments when an artifact is reused, we're using sepcialized `make` actions, located in
66+
subfolders of `.github/actions/make`. These specialized actions are parameterless, except for the github token, which
67+
they just forward. The github token is needed to use the `gh` CLI on the runner machine.
68+
69+
Any reused artifact should have such a specialized action. Whenever a new artifact is introduced in the workflow and it
70+
needs to be reused, an action should be added.

docs/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
- [Keystore Implementation](./KEYSTORE_IMPLEMENTATION.md)
66
- [FFI](./FFI.md)
77
- [Crypto Dependencies](./CRYPTO_DEPENDENCIES.md)
8+
- [CI/CD](./CI.md)
89

910
## API Documentation
1011

0 commit comments

Comments
 (0)