feat(build): use action-build-push-images #1802

joshuarli · 2025-10-06T22:13:55Z

this replaces image assembly with https://github.com/getsentry/action-build-and-push-images

the goal is to comply with the artifact management standard here: https://www.notion.so/sentry/Standard-Spec-Artifact-Management-22a8b10e4b5d80ecb6dcca3eb4558f5b

… e2e ci" This reverts commit 9c06ef6.

joshuarli · 2025-10-06T22:42:46Z

.github/workflows/image.yml

-
-  assemble-ghcr:
-    needs: [build-setup, build-image]
-    if: "needs.build-setup.outputs.full_ci == 'true'"


this is now done on every commit since e2e now needs ghcr.io/getsentry/symbolicator:${{ github.sha }} because it can't download symbolicator-amd64.tar anymore

not a big deal imo, very little overhead since actually building the final docker image is really fast

.github/workflows/image.yml

Dav1dde

I don't think this workflow works the way you set it up. It never builds the multi arch image, which a majority of the workflow did you ended up replacing. Have you checked that the output/uploaded image is actually multi arch?

Also this workflow now no longer works on forks because it needs push permissions where previously it ran fine for external contributors.

.github/workflows/image.yml

joshuarli · 2025-10-08T17:40:07Z

I don't think this workflow works the way you set it up. It never builds the multi arch image, which a majority of the workflow did you ended up replacing. Have you checked that the output/uploaded image is actually multi arch?

there's no more symbolicator-amd64/arm64 tag but that was only referenced in now deleted ci code: https://github.com/search?q=org%3Agetsentry%20symbolicator-amd64&type=code

right now the built image in ghcr has linux/amd64 and an unknown/unknown (looking into this now; i can't pull --platform=linux/arm64): https://github.com/getsentry/symbolicator/pkgs/container/symbolicator/538749404?tag=b17e8afc03c902d3eb0dcdfe90496ff93f60e4db

edit: github ui displays it as that because attestations: https://github.com/orgs/community/discussions/45969
but still not sure why i can't pull --platform=linux/arm64

edit 2: the unknown/unknown is just a glitch, in reality linux/arm64 wasn't being added - now fixed! https://github.com/getsentry/symbolicator/pkgs/container/symbolicator/538778621?tag=5de2c8ee9aafca6593611792bd90eb7cccf98e52

.github/workflows/image.yml

Co-authored-by: Hubert Deng <[email protected]>

.github/workflows/image.yml

hubertdeng123 · 2025-10-08T22:21:42Z

.github/workflows/image.yml

-
-      - *assemble
+          image_name: 'symbolicator'
+          platforms: linux/amd64,linux/arm64


Do we need multiplatform image for production? Perhaps @Dav1dde knows better here, but I guess there is no harm having it

We need multi arch for production and self hosted.

Yeah, we need to make sure multi-arch is available for self-hosted on ghcr, but are we running symbolicator on arm64 and amd64 in prod? Since this is pushing to artifact registry

but are we running symbolicator on arm64 and amd64 in prod? Since this is pushing to artifact registry

We are.

.github/workflows/image.yml

hubertdeng123 · 2025-10-08T22:25:31Z

.github/workflows/image.yml

        with:
          project_name: symbolicator
-          image_url: symbolicator-amd64
+          image_url: ghcr.io/getsentry/symbolicator:${{ github.event.pull_request.head.sha || github.sha }}


I think this will still fail on forks, we may need to ensure we're building the image on forks, saving the artifact, and loading it here to ensure it works

Something similar to https://github.com/getsentry/relay/blob/master/.github/workflows/ci.yml#L521-L531

i thought we want to skip publishing (so that CI can still pass overall) if it's a run from a fork?
(then again, we lose self hosted e2e testing)

it's okay to have the permissions and push images to GHCR then? i feel like it's fine (as long as nightly isn't published)?

We should definitely skip publishing for forks, I think I linked the wrong thing above over there, but it's still possible to build the image and save the image as an artifact to be used for the self-hosted e2e tests here even on a fork. Essentially that would require keeping the existing logic for

- name: Download Docker Image uses: actions/download-artifact@v5 with: pattern: symbolicator-image@amd64 path: /tmp - name: Load Docker Image run: docker load --input /tmp/symbolicator-amd64.tar

and adding a job to save the docker image as an artifact to be reused later. Here's the right link: https://github.com/getsentry/relay/blob/master/.github/workflows/ci.yml#L485-L503

Yeah, looks like i'll have to keep that stuff around

Dav1dde · 2025-10-09T07:45:42Z

.github/workflows/image.yml

-
-      - *assemble
+          image_name: 'symbolicator'
+          platforms: linux/amd64,linux/arm64


Have you actually tested the images you're building? You're creating a amd64 image with a arm64 binary and vice versa.

I don't think this setup can even work.

@joshuarli This is accurate, we need to ensure that the cargo build step is separate from building the docker image as right now we're publishing the multi arch image with the wrong binary twice. We need to ensure that the docker image build and publish happens once and copies the correct binary when building. This may require necessary changes to the dockerfile to make sure we're copying the right binary.

Example in relay: https://github.com/getsentry/relay/blob/468fc6fca10a0d29bb78f39149b1b3fa358758f9/Dockerfile.release#L8

@hubertdeng123 do you think it would be feasible to adjust the build action to support assembly from other images, similar to what is done here by loading multiple different architectures? It's just unfortunate that docker still doesn't provide good tools to deal with multi arch images.

The setup here was inspired from objectstore, so we'll have to do the same changes to that repo (eventually).

yeah right now both images have the amd64 binary, i need to rework this pr

docker run --platform=linux/arm64 ghcr.io/getsentry/symbolicator:55ae450aac76c48d1c12340d96282d8c0eabc09c rosetta error: failed to open elf at /lib64/ld-linux-x86-64.so.2

@Dav1dde Do you mean add logic to the composite action to take existing images and assemble them into the multiplatform manifest?

Currently there exists two processes for building multiplatform images. It's not possible for us to add assembly of images to the 2nd approach since we can't add a matrix strategy to a composite action, it can only be done from the job that calls the composite action.

Artifact build on platform specific runners, image build on one runner with emulator (relay is doing this)

action-build-and-push-images is called once and directly published multi platform image without performance degradation since the most time consuming part of building artifacts is done on the appropriate runners

requires dockerfile to handle copying the binary with the correct platform

Image build happens on platform specific runners, assembly is done manually at the end

action-build-and-push-images is called twice and builds and publishes images with suffix of either arm64 or amd64

assembly for multiplatform image can only happen after the two jobs are complete

Currently there exists two processes for building multiplatform images. It's not possible for us to add assembly of images to the 2nd approach since we can't add a matrix strategy to a composite action, it can only be done from the job that calls the composite action.

I am more thinking, let the matrix build the images (like Symbolicator/Objectstore do). And then the action-build-and-push-images can run outside of the matrix and be just passed paths to the raw images (as they are passed around via CI artifacts).

Created an item to address that, makes sense to have

joshuarli · 2025-10-15T20:40:07Z

superseded by #1805

joshuarli added 2 commits October 6, 2025 15:13

replace image assembly with action-build-push-images

5eec0bb

ref: dedupe buildx and ghcr should be full ci only, same with e2e ci

9c06ef6

joshuarli force-pushed the use-action-build-push-images branch from d12af1e to 9c06ef6 Compare October 6, 2025 22:27

joshuarli added 2 commits October 6, 2025 15:29

Revert "ref: dedupe buildx and ghcr should be full ci only, same with…

32eff4c

… e2e ci" This reverts commit 9c06ef6.

dedupe buildx

3e345bb

joshuarli commented Oct 6, 2025

View reviewed changes

joshuarli requested a review from hubertdeng123 October 6, 2025 22:43

joshuarli marked this pull request as ready for review October 6, 2025 22:43

joshuarli requested a review from a team as a code owner October 6, 2025 22:43

seer-by-sentry bot reviewed Oct 6, 2025

View reviewed changes

.github/workflows/image.yml Show resolved Hide resolved

joshuarli commented Oct 6, 2025

View reviewed changes

.github/workflows/image.yml Show resolved Hide resolved

joshuarli added 2 commits October 6, 2025 16:03

publish to ghcr on prs

204491a

[skip ci] build and push ghcr nightly on default branch

26bdfd2

joshuarli commented Oct 6, 2025

View reviewed changes

.github/workflows/image.yml Outdated Show resolved Hide resolved

fix

833926c

Dav1dde added the Trigger: Full-CI label Oct 7, 2025

Dav1dde reviewed Oct 7, 2025

View reviewed changes

.github/workflows/image.yml Outdated Show resolved Hide resolved

hubertdeng123 reviewed Oct 7, 2025

View reviewed changes

.github/workflows/image.yml Show resolved Hide resolved

joshuarli added 2 commits October 8, 2025 10:24

[skip ci] full ci correct meaning

54ed7b1

ref: actually nightly is only tagged on default branch so can dedupe

b17e8af

hubertdeng123 reviewed Oct 8, 2025

View reviewed changes

.github/workflows/image.yml Outdated Show resolved Hide resolved

joshuarli and others added 4 commits October 8, 2025 11:05

Update .github/workflows/image.yml

5de2c8e

Co-authored-by: Hubert Deng <[email protected]>

linux/amd64,linux/arm64 for prod too

fa6d944

Merge remote-tracking branch 'origin' into use-action-build-push-images

5ac1184

skip for contributor and dependabot prs

55ae450

joshuarli commented Oct 8, 2025

View reviewed changes

.github/workflows/image.yml Show resolved Hide resolved

hubertdeng123 reviewed Oct 8, 2025

View reviewed changes

Dav1dde reviewed Oct 9, 2025

View reviewed changes

joshuarli mentioned this pull request Oct 14, 2025

feat(build): replace assembly with action-build-push-images #1805

Open

joshuarli closed this Oct 15, 2025

Uh oh!

feat(build): use action-build-push-images #1802

feat(build): use action-build-push-images #1802

Uh oh!

Conversation

joshuarli commented Oct 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

joshuarli Oct 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Dav1dde left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

joshuarli commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hubertdeng123 Oct 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joshuarli commented Oct 15, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

joshuarli commented Oct 6, 2025 •

edited

Loading

joshuarli Oct 6, 2025 •

edited

Loading

joshuarli commented Oct 8, 2025 •

edited

Loading

hubertdeng123 Oct 9, 2025 •

edited

Loading