-
Notifications
You must be signed in to change notification settings - Fork 72
Automate Docker Image Build and Publish #877
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 4 commits
Commits
Show all changes
32 commits
Select commit
Hold shift + click to select a range
bc4f485
Automate Docker image builds
chaptersix 42cc84f
move the docker ignore file to the correct location
chaptersix 04e92b6
fix a few bugs
chaptersix 0e9929e
support building on pr and upgrade goreleaser
chaptersix 4667b5c
Fix semgrep issues and wf permission
chaptersix 15163e4
remove write perm from re-usable wf
chaptersix 062c9b7
update go releaser to use formats instead of format and gh wf to stop…
chaptersix 2155012
turn on caching and upload build artifacts.
chaptersix 5ecf25a
remove makefile that is not longer needed
chaptersix 8918ee7
correct alpine update command
chaptersix f6d0470
Modernize Docker build with parameterization and registry flexibility
chaptersix 363f893
Address PR feedback: simplify Docker build and fix workflow issues
chaptersix 41f7f4d
Update Dockerfile
chaptersix 98cfaff
Add explicit permissions to trigger-docs workflow
chaptersix 24bbfb8
Restore Makefile from upstream
chaptersix 5aa5d9d
remoe support for ghcr, simplify script, action version update
chaptersix 4274740
a few minor fixes
chaptersix fa17267
update action versions
chaptersix ef83ff2
simplify script again
chaptersix 4864813
fix syntax and corrections for lint issues
chaptersix 05b0021
add namespace default
chaptersix b7bda83
correct image name
chaptersix 0388341
refactor update tag workflow
chaptersix 2098e5c
remove comments and checks that are no longer needed
chaptersix a096c92
security fixes
chaptersix 90577f9
add tzdata to dockerfile
chaptersix 7b1df2e
update tag latest workflow and remove latest tagging from other code
chaptersix 4249db7
refactor to build docker images only in a manually triggered workflow.
chaptersix c31c2f9
add option to publish a docker image
chaptersix c19e100
remove go releaser builds on pr and main
chaptersix 0f2c204
fix bug. collect proper cli sha
chaptersix 7c6e522
Merge branch 'main' into alex/docker-image-re
chaptersix File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| build/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| # Detect repository owner from git remote | ||
| REPO_OWNER := $(shell git remote get-url origin | sed -E 's|.*[:/]([^/]+)/[^/]+\.git|\1|' | tr '[:upper:]' '[:lower:]') | ||
|
|
||
| # For temporalio org, default to temporalio. For others, require IMAGE_REPO to be set | ||
| ifeq ($(REPO_OWNER),temporalio) | ||
| IMAGE_REPO ?= temporalio | ||
| else | ||
| ifndef IMAGE_REPO | ||
| $(error IMAGE_REPO must be set for non-temporalio repositories. Usage: make build IMAGE_REPO=your-dockerhub-username) | ||
| endif | ||
| endif | ||
|
|
||
| CLI_SHA := $(shell git rev-parse HEAD) | ||
| IMAGE_SHA_TAG := $(shell git rev-parse --short HEAD) | ||
| IMAGE_BRANCH_TAG := $(shell git rev-parse --abbrev-ref HEAD) | ||
|
|
||
| BAKE_ENV := CLI_SHA=$(CLI_SHA) \ | ||
| IMAGE_REPO=$(IMAGE_REPO) \ | ||
| IMAGE_SHA_TAG=$(IMAGE_SHA_TAG) \ | ||
| IMAGE_BRANCH_TAG=$(IMAGE_BRANCH_TAG) | ||
|
|
||
| .PHONY: help | ||
| help: ## Display this help message | ||
| @grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | sort | awk 'BEGIN {FS = ":.*?## "}; {printf " %-20s %s\n", $$1, $$2}' | ||
|
|
||
| .PHONY: build | ||
| build: ## Build Docker images (multi-arch) | ||
| @echo "Building binaries with goreleaser..." | ||
| cd ../.. && goreleaser release --snapshot --clean --skip=publish | ||
| @echo "Building Docker images (multi-arch)..." | ||
| cd ../.. && $(BAKE_ENV) docker buildx bake --file .github/docker/docker-bake.hcl cli | ||
| @echo "" | ||
| @echo "Success! Built multi-arch images" | ||
| @echo "Note: Multi-arch images are not loaded into Docker. Use 'make build-local' to build and test locally." | ||
|
|
||
| .PHONY: build-local | ||
| build-local: ## Build and load Docker image for local testing | ||
| @echo "Building binaries with goreleaser..." | ||
| cd ../.. && goreleaser release --snapshot --clean --skip=publish | ||
| @echo "Building Docker image for local platform..." | ||
| cd ../.. && $(BAKE_ENV) docker buildx bake --file .github/docker/docker-bake.hcl --set cli.platforms=linux/amd64 --load cli | ||
| @echo "Testing Docker image..." | ||
| docker run --rm $(IMAGE_REPO)/temporal:$(IMAGE_SHA_TAG) --version | ||
| @echo "" | ||
| @echo "Success! Image: $(IMAGE_REPO)/temporal:$(IMAGE_SHA_TAG)" | ||
|
|
||
| .PHONY: clean | ||
| clean: ## Clean build artifacts | ||
| rm -rf ../../dist | ||
|
|
||
| .PHONY: update-alpine | ||
| update-alpine: ## Update Alpine base image to latest version and digest (usage: make update-alpine [ALPINE_TAG=3.22]) | ||
| @if [ -n "$(ALPINE_TAG)" ]; then \ | ||
| LATEST_TAG=$(ALPINE_TAG); \ | ||
| else \ | ||
| echo "Fetching latest Alpine version from Docker Hub..."; \ | ||
| LATEST_TAG=$$(curl -s https://registry.hub.docker.com/v2/repositories/library/alpine/tags\?page_size=100 | \ | ||
| jq -r '.results[].name' | grep -E '^3\.[0-9]+$$' | sort -V | tail -1); \ | ||
| fi && \ | ||
| echo "Alpine version: $$LATEST_TAG" && \ | ||
| DIGEST=$$(docker buildx imagetools inspect alpine:$$LATEST_TAG 2>/dev/null | grep "Digest:" | head -1 | awk '{print $$2}') && \ | ||
| DIGEST_HASH=$${DIGEST#sha256:} && \ | ||
| echo "Digest: sha256:$$DIGEST_HASH" && \ | ||
| ALPINE_FULL="alpine:$$LATEST_TAG@sha256:$$DIGEST_HASH" && \ | ||
| if sed --version 2>&1 | grep -q GNU; then \ | ||
| sed -i "s|default = \"alpine:[^\"]*\"|default = \"$$ALPINE_FULL\"|" docker-bake.hcl; \ | ||
| else \ | ||
| sed -i '' "s|default = \"alpine:[^\"]*\"|default = \"$$ALPINE_FULL\"|" docker-bake.hcl; \ | ||
| fi && \ | ||
| echo "Updated docker-bake.hcl with $$ALPINE_FULL" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| # syntax=docker/dockerfile:1 | ||
|
|
||
| ARG ALPINE_IMAGE | ||
chaptersix marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| ARG BUILDARCH | ||
|
|
||
| # Build stage - copy binaries from goreleaser output | ||
| FROM --platform=$BUILDARCH scratch AS dist | ||
| COPY dist/nix_linux_amd64_v1/temporal /dist/amd64/temporal | ||
| COPY dist/nix_linux_arm64_v8.0/temporal /dist/arm64/temporal | ||
|
|
||
| # Stage to extract CA certificates and create user files | ||
| FROM ${ALPINE_IMAGE} AS certs | ||
| RUN apk add --no-cache ca-certificates && \ | ||
| adduser -u 1000 -D temporal | ||
|
|
||
| # Final stage - minimal scratch-based image | ||
| FROM scratch | ||
|
|
||
| ARG TARGETARCH | ||
|
|
||
| # Copy CA certificates from certs stage | ||
| COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ | ||
|
|
||
| # Copy passwd and group files for non-root user | ||
| COPY --from=certs /etc/passwd /etc/passwd | ||
| COPY --from=certs /etc/group /etc/group | ||
|
|
||
| # Copy the appropriate binary for target architecture | ||
| COPY --from=dist /dist/$TARGETARCH/temporal /temporal | ||
chaptersix marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| # Run as non-root user temporal (uid 1000) | ||
| USER 1000:1000 | ||
chaptersix marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ENTRYPOINT ["/temporal"] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| variable "IMAGE_REPO" { | ||
| default = "temporalio" | ||
| } | ||
|
|
||
| variable "IMAGE_SHA_TAG" {} | ||
|
|
||
| variable "IMAGE_BRANCH_TAG" {} | ||
|
|
||
| variable "CLI_SHA" { | ||
| default = "" | ||
| } | ||
|
|
||
| variable "VERSION" { | ||
| default = "dev" | ||
| } | ||
|
|
||
| variable "TAG_LATEST" { | ||
| default = false | ||
| } | ||
|
|
||
| # Alpine base image with digest for reproducible builds | ||
| variable "ALPINE_IMAGE" { | ||
| default = "alpine:3.22@sha256:4b7ce07002c69e8f3d704a9c5d6fd3053be500b7f1c69fc0d80990c2ad8dd412" | ||
| } | ||
|
|
||
| target "cli" { | ||
| dockerfile = ".github/docker/cli.Dockerfile" | ||
| context = "." | ||
| tags = compact([ | ||
| "${IMAGE_REPO}/temporal:${IMAGE_SHA_TAG}", | ||
| "${IMAGE_REPO}/temporal:${VERSION}", | ||
| TAG_LATEST ? "${IMAGE_REPO}/temporal:latest" : "", | ||
| ]) | ||
| platforms = ["linux/amd64", "linux/arm64"] | ||
| args = { | ||
| ALPINE_IMAGE = "${ALPINE_IMAGE}" | ||
| } | ||
| labels = { | ||
| "org.opencontainers.image.title" = "temporal" | ||
| "org.opencontainers.image.description" = "Temporal CLI" | ||
| "org.opencontainers.image.url" = "https://github.com/temporalio/cli" | ||
| "org.opencontainers.image.source" = "https://github.com/temporalio/cli" | ||
| "org.opencontainers.image.licenses" = "MIT" | ||
| "org.opencontainers.image.revision" = "${CLI_SHA}" | ||
| "org.opencontainers.image.created" = timestamp() | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,161 @@ | ||
| name: Build and Publish (Reusable) | ||
|
|
||
| on: | ||
| workflow_call: | ||
| inputs: | ||
| publish: | ||
| description: 'Whether to publish the release and Docker image' | ||
| required: true | ||
| type: boolean | ||
| version: | ||
| description: 'Version tag for the release (required if publish is true)' | ||
| required: false | ||
| type: string | ||
| secrets: | ||
| DOCKER_USERNAME: | ||
| required: false | ||
| DOCKER_PASSWORD: | ||
| required: false | ||
|
|
||
| permissions: | ||
| contents: write | ||
|
|
||
| jobs: | ||
| build: | ||
| name: Build and Publish | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9 # v3.5.3 | ||
| with: | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Set up Go | ||
| uses: actions/setup-go@fac708d6674e30b6ba41289acaab6d4b75aa0753 # v4.0.1 | ||
chaptersix marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| with: | ||
| go-version-file: "go.mod" | ||
| check-latest: true | ||
|
|
||
| - name: Get build date | ||
| id: date | ||
| run: echo "::set-output name=date::$(date '+%F-%T')" | ||
|
|
||
| - name: Get build unix timestamp | ||
| id: timestamp | ||
| run: echo "::set-output name=timestamp::$(date '+%s')" | ||
|
|
||
| - name: Get git branch | ||
| id: branch | ||
| run: echo "::set-output name=branch::$(git rev-parse --abbrev-ref HEAD)" | ||
|
|
||
| - name: Get build platform | ||
| id: platform | ||
| run: echo "::set-output name=platform::$(go version | cut -d ' ' -f 4)" | ||
|
|
||
| - name: Get Go version | ||
| id: go | ||
| run: echo "::set-output name=go::$(go version | cut -d ' ' -f 3)" | ||
|
|
||
| - name: Run GoReleaser (release) | ||
| if: inputs.publish | ||
| uses: goreleaser/goreleaser-action@e435ccd777264be153ace6237001ef4d979d3a7a # v6.4.0 | ||
| with: | ||
| version: v2.12.7 | ||
| args: release | ||
| env: | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| BUILD_DATE: ${{ steps.date.outputs.date }} | ||
| BUILD_TS_UNIX: ${{ steps.timestamp.outputs.timestamp }} | ||
| GIT_BRANCH: ${{ steps.branch.outputs.branch }} | ||
| BUILD_PLATFORM: ${{ steps.platform.outputs.platform }} | ||
| GO_VERSION: ${{ steps.go.outputs.go }} | ||
|
|
||
| - name: Run GoReleaser (snapshot) | ||
| if: ${{ !inputs.publish }} | ||
| uses: goreleaser/goreleaser-action@e435ccd777264be153ace6237001ef4d979d3a7a # v6.4.0 | ||
| with: | ||
| version: v2.12.7 | ||
| args: release --snapshot --clean | ||
| env: | ||
| BUILD_DATE: ${{ steps.date.outputs.date }} | ||
| BUILD_TS_UNIX: ${{ steps.timestamp.outputs.timestamp }} | ||
| GIT_BRANCH: ${{ steps.branch.outputs.branch }} | ||
| BUILD_PLATFORM: ${{ steps.platform.outputs.platform }} | ||
| GO_VERSION: ${{ steps.go.outputs.go }} | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Log in to Docker Hub | ||
| if: inputs.publish && github.repository_owner == 'temporalio' | ||
| uses: docker/login-action@v3 | ||
| with: | ||
| username: ${{ secrets.DOCKER_USERNAME }} | ||
| password: ${{ secrets.DOCKER_PASSWORD }} | ||
|
|
||
| - name: Get build metadata | ||
| id: meta | ||
| run: | | ||
| echo "cli_sha=$(git rev-parse HEAD)" >> $GITHUB_OUTPUT | ||
chaptersix marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| echo "image_sha_tag=$(git rev-parse --short HEAD)" >> $GITHUB_OUTPUT | ||
| echo "image_branch_tag=$(git rev-parse --abbrev-ref HEAD)" >> $GITHUB_OUTPUT | ||
|
|
||
| if [[ "${{ inputs.publish }}" == "true" ]]; then | ||
| # Get version from input, strip 'v' prefix | ||
| VERSION="${{ inputs.version }}" | ||
| VERSION="${VERSION#v}" | ||
| echo "version=$VERSION" >> $GITHUB_OUTPUT | ||
| else | ||
| echo "version=snapshot" >> $GITHUB_OUTPUT | ||
| fi | ||
|
|
||
| # Determine image repo based on repository owner | ||
| if [[ "${{ github.repository_owner }}" == "temporalio" ]]; then | ||
| echo "image_repo=temporalio" >> $GITHUB_OUTPUT | ||
| else | ||
| echo "image_repo=${{ github.repository_owner }}" >> $GITHUB_OUTPUT | ||
| fi | ||
|
|
||
chaptersix marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| - name: Check if release is latest | ||
| if: inputs.publish | ||
| id: check_latest | ||
| uses: actions/github-script@v7 | ||
| with: | ||
| script: | | ||
| const { data: release } = await github.rest.repos.getReleaseByTag({ | ||
| owner: context.repo.owner, | ||
| repo: context.repo.repo, | ||
| tag: context.ref.replace('refs/tags/', '') | ||
| }); | ||
| // Tag as latest only if release is marked as latest (not pre-release) | ||
| core.setOutput('tag_latest', release.prerelease ? 'false' : 'true'); | ||
| console.log(`Release prerelease: ${release.prerelease}, tag_latest: ${!release.prerelease}`) | ||
|
|
||
| - name: Build and push Docker image | ||
| if: inputs.publish && github.repository_owner == 'temporalio' | ||
| run: | | ||
| docker buildx bake \ | ||
| --file .github/docker/docker-bake.hcl \ | ||
| --push \ | ||
| cli | ||
| env: | ||
| CLI_SHA: ${{ steps.meta.outputs.cli_sha }} | ||
| IMAGE_SHA_TAG: ${{ steps.meta.outputs.image_sha_tag }} | ||
| IMAGE_BRANCH_TAG: ${{ steps.meta.outputs.image_branch_tag }} | ||
| VERSION: ${{ steps.meta.outputs.version }} | ||
| TAG_LATEST: ${{ steps.check_latest.outputs.tag_latest }} | ||
| IMAGE_REPO: temporalio | ||
|
|
||
| - name: Build Docker image | ||
| if: ${{ !inputs.publish }} | ||
| run: | | ||
| docker buildx bake \ | ||
| --file .github/docker/docker-bake.hcl \ | ||
| cli | ||
| env: | ||
| CLI_SHA: ${{ steps.meta.outputs.cli_sha }} | ||
| IMAGE_SHA_TAG: ${{ steps.meta.outputs.image_sha_tag }} | ||
| IMAGE_BRANCH_TAG: ${{ steps.meta.outputs.image_branch_tag }} | ||
| VERSION: ${{ steps.meta.outputs.version }} | ||
| TAG_LATEST: false | ||
| IMAGE_REPO: temporalio | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| name: Build Docker Image | ||
|
|
||
| on: | ||
| pull_request: | ||
| push: | ||
| branches: | ||
| - main | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| build: | ||
| uses: ./.github/workflows/build-and-publish.yml | ||
| with: | ||
| publish: false |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.