Skip to content

Use architecture-specific runners for multi-architecture builds#3051

Merged
brianaydemir merged 5 commits intoPelicanPlatform:mainfrom
brianaydemir:build-and-test-fixes
Feb 5, 2026
Merged

Use architecture-specific runners for multi-architecture builds#3051
brianaydemir merged 5 commits intoPelicanPlatform:mainfrom
brianaydemir:build-and-test-fixes

Conversation

@brianaydemir
Copy link
Contributor

Summary

This PR aims to address point (1) of #2997 by using architecture-specific GitHub Actions runners for building Pelican's container images.

If that turns out to work well in practice, then point (2) is a straightforward change to .github/workflows/build-and-test.yml.

Point (3) is still a bit of a mystery to me, wherein I'll need to observe the behaviors of our revised GitHub Actions workflows before deciding upon further improvements. That said, I'm hoping the various changes made in this PR and #3022 will result in fewer "useless" caches.

Details

As with #3022, @jhiemstrawisc I would suggest reviewing this PR commit-by-commit, because the commit messages contain more information about what I had in mind.

I'm also more than happy to trigger, on demand (ask), a particular scenario in my own fork.

- Split the macOS and Windows workflows out into their own files because
  it's challenging trying to read through conditionals on every other step
  definition.

- Make various tweaks so that a three-way diff between test-linux.yml,
  test-macos.yml, and test-windows.yml highlights only real differences.

- On Windows, we don't do a real build of the web UI, so stop setting up
  Node.js and the Next.js cache.

- On Windows, the Go set up action no longer takes an inordinate amount of
  time, so remove the workaound.

  See the comments on actions/setup-go#515 for more.
It's a generated directory, which means it shouldn't affect the suitability
of the cache, and I've seen new(?) failures where the cache action times out
when trying to save the cache.
@brianaydemir brianaydemir force-pushed the build-and-test-fixes branch 2 times, most recently from ae74f73 to fbb5691 Compare February 5, 2026 00:18
@brianaydemir
Copy link
Contributor Author

I intended to include screenshots like these in my original write-up, but apparently I did not, so I do so now…

The changes being made here hopefully make it easier to understand what the build-and-test workflow is actually doing, especially for steps involving "matrices."

Previously

Screenshot 2026-02-04 at 18 19 57

With this PR (hopefully)

Screenshot 2026-02-04 at 18 31 27

In particular, 'setup-go' seems to install the _minimum_ version
satisfying whatever is in 'go.mod', so we need to be more explicit
about the versions that we're interested in.
- Switch to the Docker metadata action for putting together tags for the
  build-push action. Add more tags, because it's easy now.

- Fold the jobs for the testing and dev containers into the one for the
  server containers. The code duplication is not worth the maintenance cost.

- Clean up names so that they're readable in GitHub's web interface.

- Change how we interact with the GitHub Actions cache for Docker's build
  cache and the GitHub workspace so that the workflow is more resilient to
  being re-run (e.g., for flakey tests, transient errors, etc.)
@brianaydemir
Copy link
Contributor Author

Oof, GitHub's diff for test-linux.yml is atrocious: It's mostly just whitespace. On a Mac, the following is a bit clearer:

git difftool --tool opendiff main -- .github/workflows/test-linux.yml

@brianaydemir brianaydemir marked this pull request as ready for review February 5, 2026 13:53
Copy link
Member

@jhiemstrawisc jhiemstrawisc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this looks really clean, thank you for the specific commits with detailed messages. After poring over everything, I had one request for an extra comment, one question about workflow triggers and one request for a follow up after this gets merged.

Since these are all minor, I'll pre-approve and let you address them as you see fit.

@brianaydemir brianaydemir merged commit b6aff3c into PelicanPlatform:main Feb 5, 2026
16 of 17 checks passed
@brianaydemir brianaydemir deleted the build-and-test-fixes branch February 5, 2026 22:58
brianaydemir added a commit to brianaydemir/pelicanplatform-pelican that referenced this pull request Feb 7, 2026
brianaydemir added a commit to brianaydemir/pelicanplatform-pelican that referenced this pull request Feb 7, 2026
The GitHub Actions should no longer require it as of PelicanPlatform#3051. Local builds
might still need it, but the fewer one-off hacks that are incorporated into
the CI pipeline, the better.
brianaydemir added a commit to brianaydemir/pelicanplatform-pelican that referenced this pull request Feb 8, 2026
The GitHub Actions should no longer require it as of PelicanPlatform#3051. Local builds
might still need it, but the fewer one-off hacks that are incorporated into
the CI pipeline, the better.
brianaydemir added a commit to brianaydemir/pelicanplatform-pelican that referenced this pull request Feb 8, 2026
The GitHub Actions should no longer require it as of PelicanPlatform#3051.

Local builds might still need the workaround, but the fewer one-off
hacks that are incorporated into the CI pipeline, the better.
brianaydemir added a commit to brianaydemir/pelicanplatform-pelican that referenced this pull request Feb 8, 2026
GitHub Actions should no longer require it as of PelicanPlatform#3051.

Local builds might still need the workaround, but the fewer one-off
hacks that are incorporated into the CI pipeline, the better.
brianaydemir added a commit to brianaydemir/pelicanplatform-pelican that referenced this pull request Feb 9, 2026
GitHub Actions should no longer require it as of PelicanPlatform#3051.

Local builds might still need the workaround, but the fewer one-off
hacks that are incorporated into the CI pipeline, the better.
brianaydemir added a commit to brianaydemir/pelicanplatform-pelican that referenced this pull request Feb 10, 2026
GitHub Actions should no longer require it as of PelicanPlatform#3051.

Local builds might still need the workaround, but the fewer one-off
hacks that are incorporated into the CI pipeline, the better.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

container critical High priority for next release internal Internal code improvements, not user-facing test Improvements to the test suite

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve how GitHub Actions build multi-architecture container images

2 participants