ci: improve the testing of multiple configurations

[abrown]

In #586, @ncruces is running into limitations with how CI is set up for this repository: it is difficult (impossible?) to test wasi-libc when compiled with different configurations than the default. To me, the issue boils down to this kind of CI job:

wasi-libc/.github/workflows/main.yml

Lines 133 to 144 in 4720b34

	export WASI_DIR=$(realpath $($CLANG_DIR/clang -print-resource-dir)/lib/wasi/)
	export WASIP1_DIR=$(realpath $($CLANG_DIR/clang -print-resource-dir)/lib/wasip1/)
	export WASIP2_DIR=$(realpath $($CLANG_DIR/clang -print-resource-dir)/lib/wasip2/)
	mkdir -p $WASI_DIR $WASIP1_DIR $WASIP2_DIR
	cp build/download/libclang_rt.builtins-wasm32.a $WASI_DIR
	cp build/download/libclang_rt.builtins-wasm32.a $WASIP1_DIR
	cp build/download/libclang_rt.builtins-wasm32.a $WASIP2_DIR
	TARGET_TRIPLE=wasm32-wasi make test
	TARGET_TRIPLE=wasm32-wasip1 make test
	TARGET_TRIPLE=wasm32-wasip2 make test
	TARGET_TRIPLE=wasm32-wasi-threads make test
	TARGET_TRIPLE=wasm32-wasip1-threads make test

We set up the compilers, build all of the libc targets, and then test them all sequentially. I believe this is happening because:

1. setting up Clang is a bit tricky for each runner so we just want to do it once; we could fix this by creating a custom action encapsulating this CI logic
2. we need to package up the built artifacts for all of the targets into one sysroot-*.tgz file for wasi-sdk to consume; I'm not sure what to do about this but possibly wasi-sdk could retrieve more artifacts (?)

Ideally, the choice of target to build and other compiler flags should be part of the build matrix. This would (a) make it easier for us to add a new matrix line to build and test the SIMD functionality, e.g., but also (b) would parallelize the build, speeding up CI. It might also have the benefit of (c) clearly identifying which artifacts wasi-sdk pulls in, because looking at main.yml I worry that we may not always be picking up the right artifacts (e.g., enable_pic: true?). Am I missing any other rationale to why things are the way they are?

Solutions

Here are some things I've thought through, but I'm looking for others as well:

• merge repositories: we could merge wasi-sdk and wasi-libc as suggested in Should we merge wasi-libc into wasi-sdk? #427. This could significantly reduce the Clang set up since we're building it in wasi-sdk regardless. I'm not sure it totally solves the problem because we would still want to be able to test multiple configurations of wasi-libc.
• create a custom setup action: this would make it easier to add new, separate CI jobs but we likely don't want separate CI jobs
• package artifacts per target: if we create separate sysroot-*.tgz files per WASI target and per build configuration, we can both parallelize the build and easily add new build steps. I'm not sure how we would uniquely identify the build configuration, though; perhaps we hash something, or force ourselves to add an ID in main.yml. If we had that, though, would something like the following work: sysroot-{os}-clang-{clang-version}-{wasi target}-{build config id}.tgz? @alexcrichton, would we then need to understand the difference in wasi-sdk's CMake files? Or no, since it is built again there?

Any feedback is appreciated here as we think through how to enable better testing of wasi-libc.

[abrown]

It may also be that we don't need to worry about (2) up above: "package up all of the build targets..." Does anyone know anyone using the artifacts produced for each build in this repository, e.g., https://github.com/WebAssembly/wasi-libc/actions/runs/15407278884? I suspect people may be retrieving the sysroot from wasi-sdk or no?