Add TuxMake as an alternative backend for kernel builds#3037
Add TuxMake as an alternative backend for kernel builds#3037nuclearcat merged 7 commits intokernelci:mainfrom
Conversation
7634896 to
8f4848a
Compare
|
At first glance, this is a wonderful PR - thank you for the excellent work. Staging tests show no breakage; however, we should enable at least one build in the pipeline using TuxMake to properly validate it. |
8f4848a to
350db6b
Compare
361b907 to
37ddf7f
Compare
|
Also linked: kernelci/kernelci-pipeline#1406 |
4d15555 to
8793789
Compare
|
I have updated the series based on @bhcopeland feedback. |
| if self._backend == 'tuxmake': | ||
| # For TuxMake, upload everything in artifacts directory | ||
| print("[_upload_artifacts] TuxMake backend: discovering files in artifacts dir") | ||
| for root, dirs, files in os.walk(self._af_dir): |
There was a problem hiding this comment.
arm64 Image.gz not in KERNEL_IMAGE_NAMES['arm64'] = {'Image'} We need to add 'Image.gz' otherwise we get artifacts failures by mismatching
There was a problem hiding this comment.
Thank you @bhcopeland fixed in commit "kbuild: Add tuxmake backend support"
kernelci/kbuild.py
Outdated
| f"--output-dir={self._af_dir}", | ||
| ] | ||
|
|
||
| if use_kconfig_flag: |
There was a problem hiding this comment.
The issue I see here is, ChromeOS defconfig overwrites: use_kconfig_flag = False, this then will cause tuxmake to default to defconfig and then the chromeos config would be sliently disregarded
There was a problem hiding this comment.
Thank you @bhcopeland fixed in commit "kbuild: Use tuxmake native fragment support with --kconfig-add"
8793789 to
53dfb7c
Compare
|
I put together a quick validation script We need to add stage=1 before tuxmake command, _build_with_tuxmake() as clang21 was treated as an infra error instead of a build error |
53dfb7c to
5d7b6bf
Compare
|
https://staging.kernelci.org:9000/viewer?node_id=699d5e671bb1797fa875ce99 tuxmake makes dtbs.tar.xz, in metadata and as artifact, in original kbuild we upload them as separate files. |
2b6e3d7 to
241844f
Compare
|
I will set temporary staging-skip to test other PRs. |
Add tuxmake kernel build tool to the kernelci Docker fragment. Pin to version 1.35.0 for reproducibility. Reviewed-by: Ben Copeland <ben.copeland@linaro.org> Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
Add a 'backend' parameter that defaults to 'make' for backward compatibility. Extract the make build logic from _generate_script() into a dedicated _build_with_make() method so different backends can have their own build implementations. Reviewed-by: Ben Copeland <ben.copeland@linaro.org> Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
Add tuxmake as an alternative build backend, enabled with the USE_TUXMAKE=1 environment variable. Refactor _parse_fragments() to return a list of fragment file paths instead of a count and update _merge_frags() to accept this list. Add _build_with_tuxmake() method with basic tuxmake invocation. Only call _merge_frags() for the make backend since tuxmake handles fragments via --kconfig-add, added in the next commit. Reviewed-by: Ben Copeland <ben.copeland@linaro.org> Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
Use tuxmake --kconfig-add for config fragments instead of merging them manually. Add --output-dir to control where tuxmake places build artifacts. Build the dtbs target for architectures that support it. Handle multiple defconfigs by passing the first as --kconfig and the rest as --kconfig-add. Handle ChromeOS cros:// defconfig by writing the fetched config fragments to a file and passing it as --kconfig-add on top of defconfig. This lets tuxmake do the merge and olddefconfig instead of doing it before tuxmake runs. Reviewed-by: Ben Copeland <ben.copeland@linaro.org> Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
Add dtbs_check target support for the tuxmake backend. When dtbs_check is requested, run only the dtbs_check target instead of the normal kernel, modules and dtbs targets. Also skip firmware fetch for dtbs_check builds since it is not needed. Reviewed-by: Ben Copeland <ben.copeland@linaro.org> Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
241844f to
b690f6a
Compare
tuxmake outputs Image.gz for arm64 instead of Image. LAVA does not recognize 'image.gz' as kernel type and falls back to bootm which is wrong for these devices. Strip the .gz suffix from the kernel type and add compression: gz so LAVA gets the right boot method and decompresses the image. Update u-boot, barebox and grub boot templates. Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
mkbootimg expects an uncompressed Image file. When tuxmake is used as build backend the kernel is Image.gz instead of Image. Add a gunzip step before mkbootimg when the kernel type ends with .gz. Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
| image: ghcr.io/mwasilew/docker-mkbootimage:master | ||
| steps: | ||
| {%- if node.data.kernel_type.endswith('.gz') %} | ||
| - gunzip Image.gz |
There was a problem hiding this comment.
what if I don't want to use compress kernel image ?
There was a problem hiding this comment.
Both are supported, this is only if it comes through as a zipped file it will run this. TuxMake tends to produce compressed images, while the make backend produces uncompressed ones. The fastboot template adapts to whichever it receives.
|
As results on staging are good - merging it. |
TuxMake is a thin wrapper around the kernel build system that provides
consistent toolchain handling and build configuration. This series introduces a
backend selection mechanism and implements full TuxMake support including
configuration fragments, firmware handling, and dtbs_check builds.