Skip to content

Commit a4be23c

Browse files
mergify[bot]mergify[bot]
authored
captain: uv, logging, remote-Docker fixes, gha enhancements (tinkerbell#61)
#### captain: `uv`, `logging`, remote-Docker fixes, `gha` enhancements - 🍀 docker/mkosi: let /work be a regular directory, and mount specific files and dirs instead - this avoids remote-fs errors in mkosi when running against remote Docker daemon (eg: colima, Docker Desktop) - no, one can't simply mount tmpfs to `/work/mkosi.tools` -- mkosi wants to be able to delete it - ideally, `mkosi.output` (a misnomer?) would also be shared across invocations using a Volume (not yet though) - 🐸 uv: pyproject.toml: add missing version field - so one can use `uv run build.py` without worrying about deps - 🌳 docker/mkosi: add `/cache/packages` Docker Volume for download caching - this way we don't overwhelm Debian apt mirrors during development - 🌱 python: use standard logging and Rich - drop all custom logging infra, standardize on logging - introduce Rich for logging & exception handling - replace custom progressbar with Rich's - 🌿 python: switch to `uv` everywhere - switch from system-breaking `pip` to fully-contained `uv` - let `uv` install Python, don't use Debian's -- we get a single Python version for free - also: use Docker buildx explicitly; use plain output - build/docs: switch from plain Python+pip to uv; mention 3.13 instead of 3.10 - consolidate usage of pyproject.toml; drop old requirements-dev.txt and requirements.txt - docker: run in interactive mode (tty) if isatty(), pass TERM/COLUMNS env - docker: prime `uv`'s caches during Dockerfile build - 🌵 gha: better all around GHA logging with colors and wide output - gha/docker: force Rich colored 160-wide console under GHA - pass down GITHUB_ACTIONS env so children can know - gha: reduce apt verbosity drastically with `Dpkg::Use-Pty=0` - 🍃 gha: bump to cache restore/save v5 and download/upload-artifact v6 - so gha stops complaining about Node.js 20 - 🌴 README.md: show `uv` install instructions and fix log.py and __init__.py
2 parents 9ea7a56 + e93f658 commit a4be23c

31 files changed

Lines changed: 554 additions & 629 deletions

‎.github/workflows/ci.yml‎

Lines changed: 47 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -22,10 +22,8 @@ jobs:
2222
- name: Checkout code
2323
uses: actions/checkout@v6
2424

25-
- name: Set up Python
26-
uses: actions/setup-python@v5
27-
with:
28-
python-version: "3.12"
25+
- name: Install uv
26+
uses: astral-sh/setup-uv@v7
2927

3028
- name: Install lint tools
3129
run: make lint-install
@@ -49,7 +47,7 @@ jobs:
4947

5048
- name: Check amd64 kernel cache
5149
id: amd64-cache
52-
uses: actions/cache/restore@v4
50+
uses: actions/cache/restore@v5
5351
with:
5452
path: |
5553
mkosi.output/kernel/${{ env.KERNEL_VERSION }}/amd64
@@ -58,7 +56,7 @@ jobs:
5856

5957
- name: Check arm64 kernel cache
6058
id: arm64-cache
61-
uses: actions/cache/restore@v4
59+
uses: actions/cache/restore@v5
6260
with:
6361
path: |
6462
mkosi.output/kernel/${{ env.KERNEL_VERSION }}/arm64
@@ -118,7 +116,7 @@ jobs:
118116
docker tag "${REMOTE}:${HASH}-${{ matrix.arch }}" "${{ env.BUILDER_IMAGE }}"
119117
echo "built=false" >> "$GITHUB_OUTPUT"
120118
else
121-
docker build -t "${{ env.BUILDER_IMAGE }}:${HASH}" -t "${{ env.BUILDER_IMAGE }}" .
119+
docker buildx build --progress=plain -t "${{ env.BUILDER_IMAGE }}:${HASH}" -t "${{ env.BUILDER_IMAGE }}" .
122120
echo "built=true" >> "$GITHUB_OUTPUT"
123121
fi
124122
@@ -136,31 +134,31 @@ jobs:
136134

137135
- name: Restore kernel cache
138136
id: kernel-cache
139-
uses: actions/cache/restore@v4
137+
uses: actions/cache/restore@v5
140138
with:
141139
path: |
142140
mkosi.output/kernel/${{ env.KERNEL_VERSION }}/${{ matrix.arch }}
143141
key: ${{ steps.kernel-cache-key.outputs.key }}
144142

145-
- name: Install Python dependencies
146-
run: pip install -r requirements.txt
143+
- name: Install uv
144+
uses: astral-sh/setup-uv@v7
147145

148146
- name: Build kernel
149-
run: ./build.py kernel
147+
run: uv run ./build.py kernel
150148

151149
- name: Fix output file ownership
152150
run: sudo chown -R "$(id -u):$(id -g)" mkosi.output/
153151

154152
- name: Save kernel cache
155153
if: github.ref == 'refs/heads/main' && steps.kernel-cache.outputs.cache-hit != 'true'
156-
uses: actions/cache/save@v4
154+
uses: actions/cache/save@v5
157155
with:
158156
path: |
159157
mkosi.output/kernel/${{ env.KERNEL_VERSION }}/${{ matrix.arch }}
160158
key: ${{ steps.kernel-cache-key.outputs.key }}
161159

162160
- name: Upload kernel artifacts
163-
uses: actions/upload-artifact@v4
161+
uses: actions/upload-artifact@v6
164162
with:
165163
name: kernel-${{ matrix.arch }}
166164
path: |
@@ -188,30 +186,30 @@ jobs:
188186

189187
- name: Restore tools cache
190188
id: tools-cache
191-
uses: actions/cache/restore@v4
189+
uses: actions/cache/restore@v5
192190
with:
193191
path: |
194192
mkosi.output/tools/${{ matrix.arch }}/usr/local/bin
195193
mkosi.output/tools/${{ matrix.arch }}/opt/cni
196194
key: tools-${{ matrix.arch }}-${{ hashFiles('captain/tools.py') }}
197195

198-
- name: Install Python dependencies
199-
run: pip install -r requirements.txt
196+
- name: Install uv
197+
uses: astral-sh/setup-uv@v7
200198

201199
- name: Download tools
202-
run: ./build.py tools
200+
run: uv run ./build.py tools
203201

204202
- name: Save tools cache
205203
if: github.ref == 'refs/heads/main' && steps.tools-cache.outputs.cache-hit != 'true'
206-
uses: actions/cache/save@v4
204+
uses: actions/cache/save@v5
207205
with:
208206
path: |
209207
mkosi.output/tools/${{ matrix.arch }}/usr/local/bin
210208
mkosi.output/tools/${{ matrix.arch }}/opt/cni
211209
key: tools-${{ matrix.arch }}-${{ hashFiles('captain/tools.py') }}
212210

213211
- name: Upload tools artifacts
214-
uses: actions/upload-artifact@v4
212+
uses: actions/upload-artifact@v6
215213
with:
216214
name: tools-${{ matrix.arch }}
217215
path: |
@@ -238,13 +236,13 @@ jobs:
238236
uses: actions/checkout@v6
239237

240238
- name: Download kernel artifacts
241-
uses: actions/download-artifact@v4
239+
uses: actions/download-artifact@v6
242240
with:
243241
name: kernel-${{ matrix.arch }}
244242
path: mkosi.output/kernel/${{ env.KERNEL_VERSION }}/${{ matrix.arch }}
245243

246244
- name: Download tools artifacts
247-
uses: actions/download-artifact@v4
245+
uses: actions/download-artifact@v6
248246
with:
249247
name: tools-${{ matrix.arch }}
250248
path: mkosi.output/tools/${{ matrix.arch }}
@@ -257,24 +255,24 @@ jobs:
257255
chmod +x mkosi.output/tools/${{ matrix.arch }}/opt/cni/bin/*
258256
259257
- name: Refresh apt cache
260-
run: sudo apt-get update
258+
run: sudo apt-get -o "Dpkg::Use-Pty=0" update
261259

262260
- name: setup-mkosi
263261
uses: systemd/mkosi@v26
264262

265263
- name: Install bubblewrap
266264
run: |
267-
sudo apt-get update
268-
sudo apt-get install -y bubblewrap
265+
sudo apt-get -o "Dpkg::Use-Pty=0" update
266+
sudo apt-get -o "Dpkg::Use-Pty=0" install -y bubblewrap
269267
270-
- name: Install Python dependencies
271-
run: pip install -r requirements.txt
268+
- name: Install uv
269+
uses: astral-sh/setup-uv@v7
272270

273271
- name: Build initramfs
274-
run: ./build.py initramfs
272+
run: uv run ./build.py initramfs
275273

276274
- name: Upload initramfs artifacts
277-
uses: actions/upload-artifact@v4
275+
uses: actions/upload-artifact@v6
278276
with:
279277
name: initramfs-${{ matrix.arch }}
280278
path: out/
@@ -341,13 +339,13 @@ jobs:
341339
docker push "${REMOTE}:${HASH}-${{ matrix.arch }}"
342340
343341
- name: Download kernel artifacts
344-
uses: actions/download-artifact@v4
342+
uses: actions/download-artifact@v6
345343
with:
346344
name: kernel-${{ matrix.arch }}
347345
path: mkosi.output/kernel/${{ env.KERNEL_VERSION }}/${{ matrix.arch }}
348346

349347
- name: Download initramfs artifacts
350-
uses: actions/download-artifact@v4
348+
uses: actions/download-artifact@v6
351349
with:
352350
name: initramfs-${{ matrix.arch }}
353351
path: out
@@ -358,14 +356,14 @@ jobs:
358356
cp "out/initramfs-${KERNEL_VERSION}-${{ matrix.output_arch }}" \
359357
"mkosi.output/initramfs/${KERNEL_VERSION}/${{ matrix.arch }}/image.cpio.zst"
360358
361-
- name: Install Python dependencies
362-
run: pip install -r requirements.txt
359+
- name: Install uv
360+
uses: astral-sh/setup-uv@v7
363361

364362
- name: Build ISO
365-
run: ./build.py iso
363+
run: uv run ./build.py iso
366364

367365
- name: Upload ISO artifact
368-
uses: actions/upload-artifact@v4
366+
uses: actions/upload-artifact@v6
369367
with:
370368
name: iso-${{ matrix.arch }}
371369
path: out/captainos-${{ env.KERNEL_VERSION }}-${{ matrix.output_arch }}.iso
@@ -395,25 +393,25 @@ jobs:
395393
run: cat .github/config.env >> "$GITHUB_ENV"
396394

397395
- name: Download kernel artifacts
398-
uses: actions/download-artifact@v4
396+
uses: actions/download-artifact@v6
399397
with:
400398
name: kernel-${{ matrix.target }}
401399
path: mkosi.output/kernel/${{ env.KERNEL_VERSION }}/${{ matrix.target }}
402400

403401
- name: Download initramfs artifacts
404-
uses: actions/download-artifact@v4
402+
uses: actions/download-artifact@v6
405403
with:
406404
name: initramfs-${{ matrix.target }}
407405
path: out
408406

409407
- name: Download ISO artifact
410-
uses: actions/download-artifact@v4
408+
uses: actions/download-artifact@v6
411409
with:
412410
name: iso-${{ matrix.target }}
413411
path: out
414412

415-
- name: Install Python dependencies
416-
run: pip install -r requirements.txt
413+
- name: Install uv
414+
uses: astral-sh/setup-uv@v7
417415

418416
- name: Log in to GHCR
419417
uses: docker/login-action@v3
@@ -423,7 +421,7 @@ jobs:
423421
password: ${{ secrets.GITHUB_TOKEN }}
424422

425423
- name: Publish artifacts to GHCR
426-
run: ./build.py release publish
424+
run: uv run ./build.py release publish
427425

428426
# -------------------------------------------------------------------
429427
# Publish combined multi-arch image (reuses per-arch registry blobs)
@@ -445,43 +443,43 @@ jobs:
445443
run: cat .github/config.env >> "$GITHUB_ENV"
446444

447445
- name: Download kernel artifacts (amd64)
448-
uses: actions/download-artifact@v4
446+
uses: actions/download-artifact@v6
449447
with:
450448
name: kernel-amd64
451449
path: mkosi.output/kernel/${{ env.KERNEL_VERSION }}/amd64
452450

453451
- name: Download initramfs artifacts (amd64)
454-
uses: actions/download-artifact@v4
452+
uses: actions/download-artifact@v6
455453
with:
456454
name: initramfs-amd64
457455
path: out
458456

459457
- name: Download ISO artifact (amd64)
460-
uses: actions/download-artifact@v4
458+
uses: actions/download-artifact@v6
461459
with:
462460
name: iso-amd64
463461
path: out
464462

465463
- name: Download kernel artifacts (arm64)
466-
uses: actions/download-artifact@v4
464+
uses: actions/download-artifact@v6
467465
with:
468466
name: kernel-arm64
469467
path: mkosi.output/kernel/${{ env.KERNEL_VERSION }}/arm64
470468

471469
- name: Download initramfs artifacts (arm64)
472-
uses: actions/download-artifact@v4
470+
uses: actions/download-artifact@v6
473471
with:
474472
name: initramfs-arm64
475473
path: out
476474

477475
- name: Download ISO artifact (arm64)
478-
uses: actions/download-artifact@v4
476+
uses: actions/download-artifact@v6
479477
with:
480478
name: iso-arm64
481479
path: out
482480

483-
- name: Install Python dependencies
484-
run: pip install -r requirements.txt
481+
- name: Install uv
482+
uses: astral-sh/setup-uv@v7
485483

486484
- name: Log in to GHCR
487485
uses: docker/login-action@v3
@@ -491,4 +489,4 @@ jobs:
491489
password: ${{ secrets.GITHUB_TOKEN }}
492490

493491
- name: Publish combined image to GHCR
494-
run: ./build.py release publish
492+
run: uv run ./build.py release publish

‎.github/workflows/release.yml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -27,13 +27,13 @@ jobs:
2727
- name: Load shared config
2828
run: cat .github/config.env >> "$GITHUB_ENV"
2929

30-
- name: Install Python dependencies
31-
run: pip install -r requirements.txt
30+
- name: Install uv
31+
uses: astral-sh/setup-uv@v7
3232

3333
- name: Pull release artifacts (combined)
3434
env:
3535
VERSION_EXCLUDE: ${{ github.ref_name }}
36-
run: ./build.py release pull --target combined --pull-output artifacts/combined
36+
run: uv run ./build.py release pull --target combined --pull-output artifacts/combined
3737

3838
- name: Create GitHub Release
3939
env:
@@ -47,4 +47,4 @@ jobs:
4747
- name: Tag OCI artifacts with version
4848
env:
4949
VERSION_EXCLUDE: ${{ github.ref_name }}
50-
run: ./build.py release tag ${{ github.ref_name }}
50+
run: uv run ./build.py release tag ${{ github.ref_name }}

‎CONTRIBUTING.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ Please read and understand the DCO found [here](https://github.com/tinkerbell/or
1717

1818
## Environment Details
1919

20-
Building is handled by a Python script, please see the [build.py](build.py) for details. Only Python >= 3.10 and Docker are required.
20+
Building is handled by a Python script, please see the [build.py](build.py) for details. Only `uv` (Python) and Docker are required.
2121

2222
## How to Submit Change Requests
2323

‎Dockerfile‎

Lines changed: 20 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -9,11 +9,7 @@ ARG MKOSI_VERSION=v26
99
ENV DEBIAN_FRONTEND=noninteractive
1010

1111
# Install mkosi runtime dependencies and kernel build dependencies in one layer
12-
RUN apt-get update && apt-get install -y --no-install-recommends \
13-
# mkosi runtime deps
14-
python3 \
15-
python3-pip \
16-
python3-venv \
12+
RUN apt-get -o "Dpkg::Use-Pty=0" update && apt-get -o "Dpkg::Use-Pty=0" install -y --no-install-recommends \
1713
apt \
1814
dpkg \
1915
debian-archive-keyring \
@@ -59,20 +55,32 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
5955
grub-common \
6056
&& NATIVE_ARCH="$(dpkg --print-architecture)" \
6157
&& FOREIGN_ARCH=$([ "$NATIVE_ARCH" = "amd64" ] && echo "arm64" || echo "amd64") \
62-
&& apt-get install -y --no-install-recommends "grub-efi-${NATIVE_ARCH}-bin" \
58+
&& apt-get -o "Dpkg::Use-Pty=0" install -y --no-install-recommends "grub-efi-${NATIVE_ARCH}-bin" \
6359
&& dpkg --add-architecture "$FOREIGN_ARCH" \
64-
&& apt-get update \
65-
&& apt-get install -y --no-install-recommends "grub-efi-${FOREIGN_ARCH}-bin:${FOREIGN_ARCH}" \
60+
&& apt-get -o "Dpkg::Use-Pty=0" update \
61+
&& apt-get -o "Dpkg::Use-Pty=0" install -y --no-install-recommends "grub-efi-${FOREIGN_ARCH}-bin:${FOREIGN_ARCH}" \
6662
&& rm -rf /var/lib/apt/lists/*
6763

68-
# Install mkosi from GitHub (not on PyPI)
69-
RUN pip3 install --break-system-packages \
70-
configargparse \
71-
"git+https://github.com/systemd/mkosi.git@${MKOSI_VERSION}"
64+
# Install astral-sh's uv with a script - install to /usr for global access
65+
RUN curl -LsSf https://astral.sh/uv/install.sh | env UV_INSTALL_DIR="/usr/bin" sh
66+
67+
# Verify uv is functional
68+
RUN uv --version
69+
70+
# Install mkosi from GitHub (not on PyPI) via uv; symlink to /usr/bin for global access
71+
RUN uv tool install "git+https://github.com/systemd/mkosi.git@${MKOSI_VERSION}"
72+
RUN ln -sf ~/.local/bin/mkosi /usr/bin/mkosi
7273

7374
# Verify mkosi is functional
7475
RUN mkosi --version
7576

77+
# Prime uv's cache with our pyproject.toml to speed up runtime
78+
COPY pyproject.toml /tmp/pyproject.toml
79+
COPY captain /tmp/captain
80+
COPY build.py /tmp/build.py
81+
WORKDIR /tmp
82+
RUN uv --verbose run build.py --help
83+
7684
WORKDIR /work
7785
ENTRYPOINT ["mkosi"]
7886
CMD ["build"]

0 commit comments

Comments
 (0)