Skip to content

feat(build): use action-build-push-images #1802

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 13 commits into from
Closed

feat(build): use action-build-push-images #1802

Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
5eec0bb
replace image assembly with action-build-push-images
joshuarli Oct 6, 2025
9c06ef6
ref: dedupe buildx and ghcr should be full ci only, same with e2e ci
joshuarli Oct 6, 2025
32eff4c
Revert "ref: dedupe buildx and ghcr should be full ci only, same with…
joshuarli Oct 6, 2025
3e345bb
dedupe buildx
joshuarli Oct 6, 2025
204491a
publish to ghcr on prs
joshuarli Oct 6, 2025
26bdfd2
[skip ci] build and push ghcr nightly on default branch
joshuarli Oct 6, 2025
833926c
fix
joshuarli Oct 6, 2025
54ed7b1
[skip ci] full ci correct meaning
joshuarli Oct 8, 2025
b17e8af
ref: actually nightly is only tagged on default branch so can dedupe
joshuarli Oct 8, 2025
5de2c8e
Update .github/workflows/image.yml
joshuarli Oct 8, 2025
fa6d944
linux/amd64,linux/arm64 for prod too
joshuarli Oct 8, 2025
5ac1184
Merge remote-tracking branch 'origin' into use-action-build-push-images
joshuarli Oct 8, 2025
55ae450
skip for contributor and dependabot prs
joshuarli Oct 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
148 changes: 34 additions & 114 deletions .github/workflows/image.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,11 @@ jobs:
build-image:
needs: build-setup

permissions:
contents: read
packages: write # Required for GHCR
id-token: write # Required for Google Artifact Registry (GAR)

strategy:
matrix:
arch: ${{ fromJson(needs.build-setup.outputs.archs) }}
Expand Down Expand Up @@ -107,118 +112,42 @@ jobs:
name: symbolicator-debug@${{ matrix.arch }}
path: /tmp/debug-info/*

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Prepare Docker Context
run: |
mkdir docker-ctx
cp Dockerfile docker-ctx/
mv symbolicator docker-ctx/

- name: Build Image
uses: docker/build-push-action@v6
with:
context: docker-ctx/
platforms: linux/${{ matrix.arch }}
tags: symbolicator-${{ matrix.arch }}
outputs: type=docker,dest=/tmp/symbolicator-${{ matrix.arch }}.tar
push: false

- name: Upload Image
uses: actions/upload-artifact@v4
with:
name: symbolicator-image@${{ matrix.arch }}
path: /tmp/symbolicator-${{ matrix.arch }}.tar

assemble-ghcr:
needs: [build-setup, build-image]
if: "needs.build-setup.outputs.full_ci == 'true'"
Copy link
Member Author

@joshuarli joshuarli Oct 6, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is now done on every commit since e2e now needs ghcr.io/getsentry/symbolicator:${{ github.sha }} because it can't download symbolicator-amd64.tar anymore

not a big deal imo, very little overhead since actually building the final docker image is really fast


name: Assemble for Github Container Registry
runs-on: ubuntu-latest

permissions:
packages: write

env:
IMAGE: "ghcr.io/getsentry/symbolicator"

steps:
- name: Docker Login
run: docker login --username '${{ github.actor }}' --password-stdin ghcr.io <<< "$GHCR_TOKEN"
env:
GHCR_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Download Images
uses: actions/download-artifact@v5
with:
pattern: symbolicator-image@*
path: /tmp
merge-multiple: true

- &assemble
name: Assemble and Push Images
run: |
set -x

IMAGES=()
for image in /tmp/symbolicator-*.tar; do
NAME="$(basename $image .tar)"
ARCH="${NAME#*-}"
TARGET="${IMAGE}:${{ github.sha }}-${ARCH}"

docker load --input "${image}"
docker tag "${NAME}" "${TARGET}"
docker push "${TARGET}"

IMAGES+=("${TARGET}")
done

docker buildx imagetools create -t "${IMAGE}:${{ github.sha }}" "${IMAGES[@]}"

if [[ "${{ github.ref_name }}" == "master" ]]; then
docker buildx imagetools create -t "${IMAGE}:nightly" "${IMAGE}:${{ github.sha }}"
fi

assemble-ar:
needs: [build-setup, build-image]
if: "needs.build-setup.outputs.full_ci == 'true'"

name: Assemble for Google Artifact Registry
runs-on: ubuntu-latest

permissions:
contents: read
id-token: write

env:
IMAGE: "us-central1-docker.pkg.dev/sentryio/symbolicator/image"

steps:
- name: Google Auth
id: auth
uses: google-github-actions/auth@v3
with:
workload_identity_provider: projects/868781662168/locations/global/workloadIdentityPools/prod-github/providers/github-oidc-pool
service_account: [email protected]

- name: Set up Cloud SDK
uses: google-github-actions/setup-gcloud@v3
- name: Build and push ghcr image
if: "!github.event.pull_request.head.repo.fork && github.actor != 'dependabot[bot]'"
uses: getsentry/action-build-and-push-images@b172ab61a5f7eabd58bd42ce231b517e79947c01
with:
version: ">= 390.0.0"

- name: Configure Docker
run: gcloud auth configure-docker us-central1-docker.pkg.dev

- name: Download Images
uses: actions/download-artifact@v5
image_name: 'symbolicator'
platforms: linux/amd64,linux/arm64
dockerfile_path: './Dockerfile'
build_context: './docker-ctx'
ghcr: true
# we need to publish on prs for self hosted e2e tests
publish_on_pr: true
tag_nightly: false
tag_latest: false

- name: Build and push production image
if: "needs.build-setup.outputs.full_ci == 'true'"
uses: getsentry/action-build-and-push-images@b172ab61a5f7eabd58bd42ce231b517e79947c01
with:
pattern: symbolicator-image@*
path: /tmp
merge-multiple: true

- *assemble
image_name: 'symbolicator'
platforms: linux/amd64,linux/arm64
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need multiplatform image for production? Perhaps @Dav1dde knows better here, but I guess there is no harm having it

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need multi arch for production and self hosted.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we need to make sure multi-arch is available for self-hosted on ghcr, but are we running symbolicator on arm64 and amd64 in prod? Since this is pushing to artifact registry

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but are we running symbolicator on arm64 and amd64 in prod? Since this is pushing to artifact registry

We are.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you actually tested the images you're building? You're creating a amd64 image with a arm64 binary and vice versa.

I don't think this setup can even work.

Copy link
Member

@hubertdeng123 hubertdeng123 Oct 9, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@joshuarli This is accurate, we need to ensure that the cargo build step is separate from building the docker image as right now we're publishing the multi arch image with the wrong binary twice. We need to ensure that the docker image build and publish happens once and copies the correct binary when building. This may require necessary changes to the dockerfile to make sure we're copying the right binary.

Example in relay: https://github.com/getsentry/relay/blob/468fc6fca10a0d29bb78f39149b1b3fa358758f9/Dockerfile.release#L8

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hubertdeng123 do you think it would be feasible to adjust the build action to support assembly from other images, similar to what is done here by loading multiple different architectures? It's just unfortunate that docker still doesn't provide good tools to deal with multi arch images.

The setup here was inspired from objectstore, so we'll have to do the same changes to that repo (eventually).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah right now both images have the amd64 binary, i need to rework this pr

docker run --platform=linux/arm64 ghcr.io/getsentry/symbolicator:55ae450aac76c48d1c12340d96282d8c0eabc09c                      
rosetta error: failed to open elf at /lib64/ld-linux-x86-64.so.2

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Dav1dde Do you mean add logic to the composite action to take existing images and assemble them into the multiplatform manifest?

Currently there exists two processes for building multiplatform images. It's not possible for us to add assembly of images to the 2nd approach since we can't add a matrix strategy to a composite action, it can only be done from the job that calls the composite action.

  1. Artifact build on platform specific runners, image build on one runner with emulator (relay is doing this)
  • action-build-and-push-images is called once and directly published multi platform image without performance degradation since the most time consuming part of building artifacts is done on the appropriate runners
  • requires dockerfile to handle copying the binary with the correct platform
  1. Image build happens on platform specific runners, assembly is done manually at the end
  • action-build-and-push-images is called twice and builds and publishes images with suffix of either arm64 or amd64
  • assembly for multiplatform image can only happen after the two jobs are complete

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently there exists two processes for building multiplatform images. It's not possible for us to add assembly of images to the 2nd approach since we can't add a matrix strategy to a composite action, it can only be done from the job that calls the composite action.

I am more thinking, let the matrix build the images (like Symbolicator/Objectstore do). And then the action-build-and-push-images can run outside of the matrix and be just passed paths to the raw images (as they are passed around via CI artifacts).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created an item to address that, makes sense to have

dockerfile_path: './Dockerfile'
build_context: './docker-ctx'
ghcr: false
google_ar: true
# note: nightly will only be tagged if on default branch
tag_nightly: true
tag_latest: false
google_ar_image_name: us-central1-docker.pkg.dev/sentryio/symbolicator/image
google_workload_identity_provider: projects/868781662168/locations/global/workloadIdentityPools/prod-github/providers/github-oidc-pool
google_service_account: [email protected]

gocd-artifacts:
needs: [build-setup, build-image]
Expand Down Expand Up @@ -260,18 +189,9 @@ jobs:
timeout-minutes: 30

steps:
- name: Download Docker Image
uses: actions/download-artifact@v5
with:
pattern: symbolicator-image@amd64
path: /tmp

- name: Load Docker Image
run: docker load --input /tmp/symbolicator-amd64.tar

- name: Run Sentry self-hosted e2e CI
uses: getsentry/self-hosted@master
with:
project_name: symbolicator
image_url: symbolicator-amd64
image_url: ghcr.io/getsentry/symbolicator:${{ github.event.pull_request.head.sha || github.sha }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this will still fail on forks, we may need to ensure we're building the image on forks, saving the artifact, and loading it here to ensure it works

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something similar to https://github.com/getsentry/relay/blob/master/.github/workflows/ci.yml#L521-L531

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i thought we want to skip publishing (so that CI can still pass overall) if it's a run from a fork?
(then again, we lose self hosted e2e testing)

it's okay to have the permissions and push images to GHCR then? i feel like it's fine (as long as nightly isn't published)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should definitely skip publishing for forks, I think I linked the wrong thing above over there, but it's still possible to build the image and save the image as an artifact to be used for the self-hosted e2e tests here even on a fork. Essentially that would require keeping the existing logic for

- name: Download Docker Image
   uses: actions/download-artifact@v5
   with:
     pattern: symbolicator-image@amd64
     path: /tmp
  
- name: Load Docker Image
   run: docker load --input /tmp/symbolicator-amd64.tar

and adding a job to save the docker image as an artifact to be reused later. Here's the right link: https://github.com/getsentry/relay/blob/master/.github/workflows/ci.yml#L485-L503

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, looks like i'll have to keep that stuff around

CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
Loading