Skip to content

Consolidate docker image publishing in a single github workflow#6604

Merged
taylanisikdemir merged 8 commits intocadence-workflow:masterfrom
taylanisikdemir:taylan/docker_push_fix
Jan 9, 2025
Merged

Consolidate docker image publishing in a single github workflow#6604
taylanisikdemir merged 8 commits intocadence-workflow:masterfrom
taylanisikdemir:taylan/docker_push_fix

Conversation

@taylanisikdemir
Copy link
Member

@taylanisikdemir taylanisikdemir commented Jan 9, 2025

Current State

Currently we have multiple CI jobs doing docker pushes.

Buildkite job pushes following images on each master push:

  • ubercadence/server:master
  • ubercadence/server:master-auto-setup
  • ubercadence/cli:master
  • ubercadence/cadence-bench:master
  • ubercadence/cadence-canary:master

Github workflow is pushing following images on each release:

  • ubercadence/server:v1.x.y
  • ubercadence/server::v1.x.y-auto-setup

Problems:

  • Github workflow is using an old version of docker push action which is broken. We have been manually pushing after each release.
  • Github workflow doesn't push cli image. It was last pushed 1 year ago.
  • Dockerhub secrets have to be maintained/rotated in multiple places: github secrets and buildkite secrets

Changes:

  • To avoid maintaining multiple CI frameworks doing similar stuff, I am deleting docker steps in from buildkite and rewriting the github workflow to do all the push/validation needed.
  • The new github workflow will push images both on master merges and on releases.
  • It will also build docker images on pull requests without pushing to validate docker build works.
  • I am getting rid of latestRelease/latestrelease image tags. We don't refer to them in any documentation. The capitalization in the name doesn't make sense. There was an attempt to fix it but due to push action being broken we never published the proper named one. We can introduce "latest" tag after this cleanup.
  • No longer pushing cadence-bench and cadence-canary images.

Testing

By triggering new workflow as part of this PR. Added a check-meta job to double check conditional expressions. I will delete it later.

After this PR merges, I will validate the master image pushes are working as expected.
After next release, I will also validate the same for release tagged images.

Comment on lines +86 to +88
# TODO: Uncomment below once image tags are validated after merging
# push: ${{ needs.meta.outputs.push_enabled == 'true' }}
push: false
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After merging this PR I will double check the image tags and then enable pushes in a follow up PR

password: ${{ secrets.CADENCE_SERVER_DOCKERHUB_TOKEN }}
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, I must have missed this step when I tried to publish multi-plat images, I was getting buildx not supported errors

@taylanisikdemir taylanisikdemir merged commit 8e1c0e4 into cadence-workflow:master Jan 9, 2025
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Comments