Skip to content

ci: linux/amd64 and linux/arm64 multi platform image #5799

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
+20 −4
Open

ci: linux/amd64 and linux/arm64 multi platform image #5799

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f733d91
ci: linux/amd64 and linux/arm64 multi platform image
sozercan Nov 8, 2024
4ab87f4
buildx
sozercan Nov 13, 2024
aeb6d99
Merge branch 'master' into multi-platform
sozercan Jul 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions kustomize.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
# Copyright 2022 The Kubernetes Authors.
# SPDX-License-Identifier: Apache-2.0

# build
FROM public.ecr.aws/docker/library/golang:1.22.7-bullseye AS builder
FROM --platform=${BUILDPLATFORM} public.ecr.aws/docker/library/golang:1.22.7-bullseye as builder
ARG VERSION
ARG DATE
ARG TARGETARCH
ARG TARGETOS
RUN mkdir /build
ADD . /build/
WORKDIR /build/kustomize
RUN CGO_ENABLED=0 GO111MODULE=on go build \
RUN CGO_ENABLED=0 GO111MODULE=on GOOS=${TARGETOS} GOARCH=${TARGETARCH} go build \
-ldflags="-s -X sigs.k8s.io/kustomize/api/provenance.version=${VERSION} \
-X sigs.k8s.io/kustomize/api/provenance.buildDate=${DATE}"

Expand Down
17 changes: 16 additions & 1 deletion releasing/cloudbuild_kustomize_image.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,26 @@ steps:
- "PROJECT_ID=$PROJECT_ID"
- "_GIT_TAG=$_GIT_TAG"
- "_PULL_BASE_REF=$_PULL_BASE_REF"
# We need to use bash to configure the build date and version properly.
- name: "gcr.io/cloud-builders/docker"
entrypoint: /bin/bash
args:
- -c
- >
docker
buildx
create
--name
builder
--use
--bootstrap
# We need to use bash to configure the build date and version properly.
- name: "gcr.io/cloud-builders/docker"
entrypoint: /bin/bash
args:
- -c
- >
docker
buildx
build
-t
gcr.io/$PROJECT_ID/kustomize:${_GIT_TAG}
Expand All @@ -27,6 +40,8 @@ steps:
VERSION=${_PULL_BASE_REF}
--build-arg
DATE=$(date -u +%FT%TZ)
--platform
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like there is no --platform option for docker build.
Are you trying to use the buildx command?

kobdotsh@instance-2:~$ sudo docker run -it --entrypoint=/bin/bash --rm gcr.io/cloud-builders/docker
root@9313da0b0c4d:/# docker build --help

Usage:  docker build [OPTIONS] PATH | URL | -

Build an image from a Dockerfile

Options:
      --add-host list           Add a custom host-to-IP mapping (host:ip)
      --build-arg list          Set build-time variables
      --cache-from strings      Images to consider as cache sources
      --cgroup-parent string    Optional parent cgroup for the container
      --compress                Compress the build context using gzip
      --cpu-period int          Limit the CPU CFS (Completely Fair Scheduler)
                                period
      --cpu-quota int           Limit the CPU CFS (Completely Fair Scheduler)
                                quota
  -c, --cpu-shares int          CPU shares (relative weight)
      --cpuset-cpus string      CPUs in which to allow execution (0-3, 0,1)
      --cpuset-mems string      MEMs in which to allow execution (0-3, 0,1)
      --disable-content-trust   Skip image verification (default true)
  -f, --file string             Name of the Dockerfile (Default is
                                'PATH/Dockerfile')
      --force-rm                Always remove intermediate containers
      --iidfile string          Write the image ID to the file
      --isolation string        Container isolation technology
      --label list              Set metadata for an image
  -m, --memory bytes            Memory limit
      --memory-swap bytes       Swap limit equal to memory plus swap: '-1' to
                                enable unlimited swap
      --network string          Set the networking mode for the RUN
                                instructions during build (default "default")
      --no-cache                Do not use cache when building the image
      --pull                    Always attempt to pull a newer version of the
                                image
  -q, --quiet                   Suppress the build output and print image ID
                                on success
      --rm                      Remove intermediate containers after a
                                successful build (default true)
      --security-opt strings    Security options
      --shm-size bytes          Size of /dev/shm
      --squash                  Squash newly built layers into a single new layer
  -t, --tag list                Name and optionally a tag in the 'name:tag' format
      --target string           Set the target build stage to build.
      --ulimit ulimit           Ulimit options (default [])

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@koba1t good catch! newer versions of docker has build automatically use buildx build, looks like the GCR image has an older version. I don't have a cloud build environment to test this, would you be able to verify this works? Thanks!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sozercan
I believe you can use gcr.io/cloud-builders/docker container image on your x86_64 computer to test that build command.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to this doc, we need to set up QEMU for building an ARM image.
https://cloud.google.com/kubernetes-engine/docs/how-to/build-multi-arch-for-arm

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@koba1t go supports cross-platform building, you don't need qemu to build an arm image. you'll need qemu to run a non-host platform image though, but we are not running any images in this context.

Copy link
Member

@koba1t koba1t Jul 10, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sozercan

The current Dockerfile is running commands like apk add. While it's possible for the Go runtime to directly generate arm64 binaries, I don't think it's feasible to create an arm64 container image without an architecture conversion layer.

kustomize/kustomize.Dockerfile

Line 18 in f9ab532

RUN apk add --no-cache git openssh

If the build is successful in your Google Cloud environment, could you please share those logs?

Copy link

@harryzcy harryzcy Jul 10, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the build is successful in your Google Cloud environment, could you please share those logs?

It's possible that build is just fine since everything installs under amd64, but kustomize will failed to run git later on during runtime. Without QEMU, does buildx actually grabs the linux/arm64 version of alpine?

linux/amd64,linux/arm64
.

images:
Expand Down