Make CI scripts usable from CLI

[nyurik]

GitHub actions are great for testing things, but at times they are tricky to reproduce locally - which means locally one has to use a different method of testing, trying to match what CI does. A common pattern I have seen is to use just - a modern and clean alternative to make. This way, CI would call exactly the same scripts as developers can use locally (if they want to). Or developers can continue using any current approach that works for them.

This would mean moving many reusable scripts to the root justfile as individual recipes. For example,

atsamd/.github/workflows/build-hal.yml

Lines 37 to 42 in e0f3cc6

	- name: Build HAL for ${{ matrix.pac }}
	run: |
	set -ex
	features=$(cat ./crates.json | jq -Mr --arg pac "${{matrix.pac}}" -c '.hal_build_variants["${{matrix.pac}}"].features | join(",")')
	target=$(cat ./crates.json | jq -Mr --arg pac "${{matrix.pac}}" -c '.hal_build_variants["${{matrix.pac}}"].target')
	cargo clippy --features=${features} --target=${target} --manifest-path=./hal/Cargo.toml -- -D warnings

can be written as this, with possibly a few more adjustments as needed:

ci-build-hal pac:
  set -ex
  features=$(cat ./crates.json | jq -Mr --arg pac "${{pac}}" -c '.hal_build_variants["${{pac}}"].features | join(",")')
  target=$(cat ./crates.json | jq -Mr --arg pac "${{pac}}" -c '.hal_build_variants["${{pac}}"].target')
  cargo clippy --features=${features} --target=${target} --manifest-path=./hal/Cargo.toml -- -D warnings

and CI would simply run just ci-build-hal {{matrix.pac}}. This requires just to be installed, but the pattern is so common that just is available as part of taiki-e/install-action (other tools can be installed with the same command, e.g. 'just,cargo-llvm-cov')

      - uses: taiki-e/install-action@v2
        with: { tool: just }

The added bonus of this is that any other, non-ci related scripts can go into the same justfile

[jbeaurivage]

That's a good suggestion, we have accumulated quite a few scripts here and there over the years. A bit of standardization probably wouldn't hurt there.

[ianrrees]

TBH, I haven't found the problem (developing/debugging CI locally) to be significant and I'm hesitant to add another language to the mix - especially when the leading contender for #830 involves adding a Rust tool to help manage the project.

Changing the GH Actions to just call some other scripting tool also doesn't help one of the more common issues with CI IME, which is that in CI, the script is run in a container which has a different set of tools etc from the development machine.

GitLab makes it fairly easy to run CI locally, I presume GitHub has similar tooling available as well. From a quick search, https://github.com/nektos/act looks like a nice option.