Skip to content

Commit 947a810

Browse files
authored
Reproducible artifact hashes for base builder images (#766)
I was looking at the build pipeline for the distributions, and was surprised to find the builder image artifacts having different SHA256 hashes across runs. e.g. compare the output of the "Upload Docker Image" step of two jobs: https://github.com/chowder/python-build-standalone/actions/runs/17012490960/job/48230404008 https://github.com/chowder/python-build-standalone/actions/runs/17012490960/job/48230532790 For the same image: ``` SHA256 digest of uploaded artifact zip is d47e11cc1c0e648bed6ec64a3613fa166c317981d252085c241c05745604973b Finalizing artifact upload Artifact image-build-linux_x86_64.zip successfully finalized. Artifact ID 3780654566 Artifact image-build-linux_x86_64 has been successfully uploaded! Final size is 80890477 bytes. Artifact ID is 3780654566 ``` ``` SHA256 digest of uploaded artifact zip is 3341d64b1d7efb680db3e4d62ae3dab9bf24e22e384f585aaf3a5b551aee5e7b Finalizing artifact upload Artifact image-build-linux_x86_64.zip successfully finalized. Artifact ID 3780667100 Artifact image-build-linux_x86_64 has been successfully uploaded! Final size is 80890477 bytes. Artifact ID is 3780667100 ``` Turns out this is because the `upload-artifact` step applies a layer of compression to the specified files, but doesn't reset/zero out the mtime, thus making these artifacts non-reproducible. Given that the docker images are already compressed [in a prior step](https://github.com/astral-sh/python-build-standalone/blob/main/.github/workflows/linux.yml#L117-L120): ```yaml - name: Compress Image run: | echo ${STEPS_BUILD_IMAGE_OUTPUTS_IMAGEID} > build/image-${MATRIX_NAME}.linux_${MATRIX_ARCH} zstd -v -T0 -6 --rm build/image-*.tar ``` We can skip the compression in the `upload-artifact` step, and get reproducible artifacts each time. #### Test Evidence See these two runs that include the proposed changes: https://github.com/chowder/python-build-standalone/actions/runs/17012517183/job/48230459046 https://github.com/chowder/python-build-standalone/actions/runs/17012517183/job/48230772691 Both generate the same digest: ``` Finished uploading artifact content to blob storage! SHA256 digest of uploaded artifact zip is c73f01690d74c103f92bd8113ff158e8b8d2d4de19dca96d36b3d6fbc688fe16 Finalizing artifact upload Artifact image-build-linux_x86_64.zip successfully finalized. Artifact ID 3780661038 Artifact image-build-linux_x86_64 has been successfully uploaded! Final size is 81208314 bytes. Artifact ID is 3780661038 ``` ``` SHA256 digest of uploaded artifact zip is c73f01690d74c103f92bd8113ff158e8b8d2d4de19dca96d36b3d6fbc688fe16 Finalizing artifact upload Artifact image-build-linux_x86_64.zip successfully finalized. Artifact ID 3780693018 Artifact image-build-linux_x86_64 has been successfully uploaded! Final size is 81208314 bytes. Artifact ID is 3780693018 ```
1 parent 3890439 commit 947a810

File tree

1 file changed

+4
-0
lines changed

1 file changed

+4
-0
lines changed

.github/workflows/linux.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,8 @@ jobs:
9797
- name: Build Image
9898
id: build-image
9999
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
100+
env:
101+
SOURCE_DATE_EPOCH: 0
100102
with:
101103
context: .
102104
file: build/${{ matrix.name }}.Dockerfile
@@ -118,6 +120,7 @@ jobs:
118120
run: |
119121
echo ${STEPS_BUILD_IMAGE_OUTPUTS_IMAGEID} > build/image-${MATRIX_NAME}.linux_${MATRIX_ARCH}
120122
zstd -v -T0 -6 --rm build/image-*.tar
123+
touch -t 197001010000 build/image-*
121124
env:
122125
STEPS_BUILD_IMAGE_OUTPUTS_IMAGEID: ${{ steps.build-image.outputs.imageid }}
123126
MATRIX_NAME: ${{ matrix.name }}
@@ -128,6 +131,7 @@ jobs:
128131
with:
129132
name: image-${{ matrix.name }}-linux_${{ matrix.arch }}
130133
path: build/image-*
134+
compression-level: '0'
131135

132136
generate-matrix:
133137
name: Generate build matrix

0 commit comments

Comments
 (0)