Skip to content

feat(build): use action-build-push-images #1802

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 13 commits into from
Closed

feat(build): use action-build-push-images #1802

Changes from 7 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
5eec0bb
replace image assembly with action-build-push-images
joshuarli Oct 6, 2025
9c06ef6
ref: dedupe buildx and ghcr should be full ci only, same with e2e ci
joshuarli Oct 6, 2025
32eff4c
Revert "ref: dedupe buildx and ghcr should be full ci only, same with…
joshuarli Oct 6, 2025
3e345bb
dedupe buildx
joshuarli Oct 6, 2025
204491a
publish to ghcr on prs
joshuarli Oct 6, 2025
26bdfd2
[skip ci] build and push ghcr nightly on default branch
joshuarli Oct 6, 2025
833926c
fix
joshuarli Oct 6, 2025
54ed7b1
[skip ci] full ci correct meaning
joshuarli Oct 8, 2025
b17e8af
ref: actually nightly is only tagged on default branch so can dedupe
joshuarli Oct 8, 2025
5de2c8e
Update .github/workflows/image.yml
joshuarli Oct 8, 2025
fa6d944
linux/amd64,linux/arm64 for prod too
joshuarli Oct 8, 2025
5ac1184
Merge remote-tracking branch 'origin' into use-action-build-push-images
joshuarli Oct 8, 2025
55ae450
skip for contributor and dependabot prs
joshuarli Oct 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
157 changes: 44 additions & 113 deletions .github/workflows/image.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,11 @@ jobs:
build-image:
needs: build-setup

permissions:
contents: read
packages: write # Required for GHCR
id-token: write # Required for Google Artifact Registry (GAR)

strategy:
matrix:
arch: ${{ fromJson(needs.build-setup.outputs.archs) }}
Expand Down Expand Up @@ -107,122 +112,57 @@ jobs:
name: symbolicator-debug@${{ matrix.arch }}
path: /tmp/debug-info/*

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Prepare Docker Context
run: |
mkdir docker-ctx
cp Dockerfile docker-ctx/
mv symbolicator docker-ctx/

- name: Build Image
uses: docker/build-push-action@v6
- name: Build and push ghcr image
uses: getsentry/action-build-and-push-images@a53f146fc1ea3cb404f2dcf7378f5b60dd98d3ca
with:
context: docker-ctx/
image_name: 'symbolicator'
platforms: linux/${{ matrix.arch }}
tags: symbolicator-${{ matrix.arch }}
outputs: type=docker,dest=/tmp/symbolicator-${{ matrix.arch }}.tar
push: false

- name: Upload Image
uses: actions/upload-artifact@v4
with:
name: symbolicator-image@${{ matrix.arch }}
path: /tmp/symbolicator-${{ matrix.arch }}.tar

assemble-ghcr:
needs: [build-setup, build-image]
if: "needs.build-setup.outputs.full_ci == 'true'"
Copy link
Member Author

@joshuarli joshuarli Oct 6, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is now done on every commit since e2e now needs ghcr.io/getsentry/symbolicator:${{ github.sha }} because it can't download symbolicator-amd64.tar anymore

not a big deal imo, very little overhead since actually building the final docker image is really fast


name: Assemble for Github Container Registry
runs-on: ubuntu-latest

permissions:
packages: write

env:
IMAGE: "ghcr.io/getsentry/symbolicator"

steps:
- name: Docker Login
run: docker login --username '${{ github.actor }}' --password-stdin ghcr.io <<< "$GHCR_TOKEN"
env:
GHCR_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Download Images
uses: actions/download-artifact@v5
dockerfile_path: './Dockerfile'
build_context: './docker-ctx'
ghcr: true
# we need to publish on prs for self hosted e2e tests
publish_on_pr: true
tag_nightly: false
tag_latest: false

- name: Build and push nightly ghcr image
# "nightly" has actually been "latest" for a while now, shrug
if: github.ref_name == github.event.repository.default_branch
uses: getsentry/action-build-and-push-images@a53f146fc1ea3cb404f2dcf7378f5b60dd98d3ca
with:
pattern: symbolicator-image@*
path: /tmp
merge-multiple: true

- &assemble
name: Assemble and Push Images
run: |
set -x

IMAGES=()
for image in /tmp/symbolicator-*.tar; do
NAME="$(basename $image .tar)"
ARCH="${NAME#*-}"
TARGET="${IMAGE}:${{ github.sha }}-${ARCH}"

docker load --input "${image}"
docker tag "${NAME}" "${TARGET}"
docker push "${TARGET}"

IMAGES+=("${TARGET}")
done

docker buildx imagetools create -t "${IMAGE}:${{ github.sha }}" "${IMAGES[@]}"

if [[ "${{ github.ref_name }}" == "master" ]]; then
docker buildx imagetools create -t "${IMAGE}:nightly" "${IMAGE}:${{ github.sha }}"
fi

assemble-ar:
needs: [build-setup, build-image]
if: "needs.build-setup.outputs.full_ci == 'true'"

name: Assemble for Google Artifact Registry
runs-on: ubuntu-latest

permissions:
contents: read
id-token: write

env:
IMAGE: "us-central1-docker.pkg.dev/sentryio/symbolicator/image"

steps:
- name: Google Auth
id: auth
uses: google-github-actions/auth@v3
with:
workload_identity_provider: projects/868781662168/locations/global/workloadIdentityPools/prod-github/providers/github-oidc-pool
service_account: [email protected]

- name: Set up Cloud SDK
uses: google-github-actions/setup-gcloud@v3
with:
version: ">= 390.0.0"

- name: Configure Docker
run: gcloud auth configure-docker us-central1-docker.pkg.dev

- name: Download Images
uses: actions/download-artifact@v5
image_name: 'symbolicator'
platforms: linux/${{ matrix.arch }}
dockerfile_path: './Dockerfile'
build_context: './docker-ctx'
ghcr: true
tag_nightly: true
tag_latest: false

- name: Build and push production image
if: github.ref_name == github.event.repository.default_branch
uses: getsentry/action-build-and-push-images@a53f146fc1ea3cb404f2dcf7378f5b60dd98d3ca
with:
pattern: symbolicator-image@*
path: /tmp
merge-multiple: true

- *assemble
image_name: 'symbolicator'
platforms: linux/${{ matrix.arch }}
dockerfile_path: './Dockerfile'
build_context: './docker-ctx'
ghcr: false
google_ar: true
tag_nightly: false
tag_latest: false
google_ar_image_name: us-central1-docker.pkg.dev/sentryio/symbolicator/image
google_workload_identity_provider: projects/868781662168/locations/global/workloadIdentityPools/prod-github/providers/github-oidc-pool
google_service_account: [email protected]

gocd-artifacts:
needs: [build-setup, build-image]
if: "needs.build-setup.outputs.full_ci == 'true'"
if: github.ref_name == github.event.repository.default_branch

name: Upload gocd artifacts
runs-on: ubuntu-latest
Expand Down Expand Up @@ -260,18 +200,9 @@ jobs:
timeout-minutes: 30

steps:
- name: Download Docker Image
uses: actions/download-artifact@v5
with:
pattern: symbolicator-image@amd64
path: /tmp

- name: Load Docker Image
run: docker load --input /tmp/symbolicator-amd64.tar

- name: Run Sentry self-hosted e2e CI
uses: getsentry/self-hosted@master
with:
project_name: symbolicator
image_url: symbolicator-amd64
image_url: ghcr.io/getsentry/symbolicator:${{ github.event.pull_request.head.sha || github.sha }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this will still fail on forks, we may need to ensure we're building the image on forks, saving the artifact, and loading it here to ensure it works

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something similar to https://github.com/getsentry/relay/blob/master/.github/workflows/ci.yml#L521-L531

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i thought we want to skip publishing (so that CI can still pass overall) if it's a run from a fork?
(then again, we lose self hosted e2e testing)

it's okay to have the permissions and push images to GHCR then? i feel like it's fine (as long as nightly isn't published)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should definitely skip publishing for forks, I think I linked the wrong thing above over there, but it's still possible to build the image and save the image as an artifact to be used for the self-hosted e2e tests here even on a fork. Essentially that would require keeping the existing logic for

- name: Download Docker Image
   uses: actions/download-artifact@v5
   with:
     pattern: symbolicator-image@amd64
     path: /tmp
  
- name: Load Docker Image
   run: docker load --input /tmp/symbolicator-amd64.tar

and adding a job to save the docker image as an artifact to be reused later. Here's the right link: https://github.com/getsentry/relay/blob/master/.github/workflows/ci.yml#L485-L503

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, looks like i'll have to keep that stuff around

CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
Loading