chore(pipeline/multiarch): don't rebuild images of architectures already built elsewhere #2155

lemeurherve · 2025-12-24T21:05:43Z

This PR adds buildarch-% and listarch-% targets to the Makefile, allowing then to build in parallel images of architectures not already built in other stages, to improve overall build duration.

Before, multiarch stage was building the following targets and architectures:

alpine_jdk17 (amd64)
alpine_jdk21 (amd64, arm64)
debian-slim_jdk17 (amd64)
debian-slim_jdk21 (amd64, arm64)
debian_jdk17 (amd64, arm64, s390x, ppc64le)
debian_jdk21 (amd64, arm64, s390x, ppc64le) 
rhel_ubi9_jdk17 (amd64, arm64, ppc64le)
rhel_ubi9_jdk21 (amd64, arm64, ppc64le)

Total: 20 distinct images in one stage.

With this change, only the following targets and architectures are built in multiarch stages:

alpine_jdk21 (arm64)
debian-slim_jdk21 (arm64)
debian_jdk17 (arm64, s390x, ppc64le)
debian_jdk21 (arm64, s390x, ppc64le)
rhel_ubi9_jdk17 (arm64, ppc64le)
rhel_ubi9_jdk21 (arm64, ppc64le)

Total: 12 distinct images.

6 in arm64 stage
2 in s390x stage
4 in ppc64le stage

Ref:

Optimise multiarch sanity check duration #2154

Testing done

List of targets per architecture

$ make listarch-amd64
alpine_jdk17
alpine_jdk21
debian-slim_jdk17
debian-slim_jdk21
debian_jdk17
debian_jdk21
rhel_ubi9_jdk17
rhel_ubi9_jdk21

$ make listarch-arm64
alpine_jdk21
debian-slim_jdk21
debian_jdk17
debian_jdk21
rhel_ubi9_jdk17
rhel_ubi9_jdk21

$ make listarch-s390x
debian_jdk17
debian_jdk21

$ make listarch-ppc64le
debian_jdk17
debian_jdk21
rhel_ubi9_jdk17
rhel_ubi9_jdk21

CI build before:

CI build after (outdated, see below):

With f453ad3, the arm64 stage went from 25 minutes to 2 minutes 27 seconds 🚀

Before f453ad3:

After:

The bottleneck is now (again 😅) the Windows stage*, the overall build duration is decreasing from 47 minutes to 23 minutes on ci.jenkins.io 🎉

I'll open an issue to discuss the split of the publish stage of trusted.ci.jenkins.io into similar parallel stages with one architecture each to reduce its build duration too.

Submitter checklist

Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
Ensure that the pull request title represents the desired changelog entry
Please describe what you did
Link to relevant issues in GitHub or Jira
Link to relevant pull requests, esp. upstream and downstream changes
Ensure you have provided tests that demonstrate the feature works or the issue is fixed

dduportal · 2025-12-25T07:04:47Z

Overall build duration decreased from 47min to 28min on ci.jenkins.io 🥳
👏

Jenkinsfile

…ady built elsewhere

…appear from pipeline graph view (see build number 2)

…ady known when entering the node() block and to avoid complexity

…other architectures)

…`make`

lemeurherve · 2025-12-25T16:17:09Z

Makefile

 else
 	@docker buildx create --use --bootstrap --driver docker-container
 endif
+# There is only an amd64 qemu image


From a replay: https://ci.jenkins.io/job/Packaging/job/docker/view/change-requests/job/PR-2155/9/pipeline-overview/?start-byte=0&selected-node=309#log-309-32

17:06:31 Status: Downloaded newer image for multiarch/qemu-user-static:latest 17:06:33 WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

Jenkinsfile

lemeurherve · 2025-12-25T16:34:19Z

With f453ad3, the arm64 stage went from 25 minutes to 2 minutes 27 seconds 🚀

Before:

After:

The bottleneck is now (again 😅) the Windows stage*, the overall build duration is decreasing from 47 minutes to 23 minutes on ci.jenkins.io 🎉

*: pending completion of jenkinsci/docker-agents#1068 that should decrease its duration.

…lacking `make`" This reverts commit 3b7877d.

lemeurherve · 2025-12-25T21:13:23Z

Last build using s390x permanent agent (from ~7min30s to ~1min 🚀):

timja · 2025-12-27T17:15:47Z

Jenkinsfile

                def imageToBuild = i

                builds[imageToBuild] = {
-                    nodeWithTimeout('docker') {


this is now different to the publish build isn't it?

The production build uses https://github.com/lemeurherve/docker/blob/dc9309cbe33f3523f7dae6c8ca4642f1145950c1/Jenkinsfile#L193

So if there's an issue in QEMU emulation we won't catch it on CI. This has happened before...

TBH, only the label was common before, nothing was preventing agent definitions to diverge between ci.jenkins.io and trusted.ci.jenkins.io.

yes but QEMU is important that its validated on CI especially for new images

I was thinking about using agents with dedicated architectures for publication on trusted.ci.jenkins.io like the ones used on ci.jenkins.io build introduced with this PR, would that be sufficient to alleviate this potential issue in your opinion?

Yes that’s fine as long as the same across both

…ady built elsewhere (#2155)

lemeurherve added chore multi-arch-support linux labels Dec 24, 2025

lemeurherve marked this pull request as ready for review December 24, 2025 21:23

lemeurherve requested a review from a team as a code owner December 24, 2025 21:23

This was referenced Dec 24, 2025

chore(pipeline/multiarch): build only the default JDK #2156

Closed

Optimise multiarch sanity check duration #2154

Open

dduportal reviewed Dec 25, 2025

View reviewed changes

Jenkinsfile Outdated Show resolved Hide resolved

lemeurherve marked this pull request as draft December 25, 2025 16:02

lemeurherve added 8 commits December 25, 2025 17:15

chore(pipeline/multiarch): don't rebuild images of architectures alre…

dfeca56

…ady built elsewhere

use intermediate variable (otherwise it's always the same value)

7911edc

list targets concerned by the architecture before building them

fc4b146

don't mark stage as NOT_BUILT as it makes the overall build steps dis…

e3acf6a

…appear from pipeline graph view (see build number 2)

s/list-%/listarch-%/ & s/archbuild-%/buildarch-%/

0d4b32b

using architecturesAndCiJioAgentLabels to ensure the CPU arch is alre…

f453ad3

…ady known when entering the node() block and to avoid complexity

setup qemu in docker-init only on amd64 (there is no image for the …

4910e05

…other architectures)

use amd64 agent for x390x for now as this permanent agent is lacking …

3b7877d

…`make`

lemeurherve force-pushed the multiarch-optim branch from ad1b1eb to 3b7877d Compare December 25, 2025 16:15

lemeurherve commented Dec 25, 2025

View reviewed changes

Jenkinsfile Outdated Show resolved Hide resolved

lemeurherve mentioned this pull request Dec 25, 2025

[ci.jenkins.io] Ensure s390x permanent agent has every required tools to build docker images jenkins-infra/helpdesk#4928

Closed

lemeurherve marked this pull request as ready for review December 25, 2025 16:26

Merge branch 'master' into multiarch-optim

5286a04

lemeurherve requested review from a team and dduportal December 25, 2025 16:42

Revert "use amd64 agent for x390x for now as this permanent agent is …

dc9309c

…lacking `make`" This reverts commit 3b7877d.

dduportal approved these changes Dec 26, 2025

View reviewed changes

lemeurherve merged commit bb0fa3c into jenkinsci:master Dec 26, 2025
11 of 12 checks passed

lemeurherve deleted the multiarch-optim branch December 26, 2025 16:17

timja reviewed Dec 27, 2025

View reviewed changes

lemeurherve added a commit that referenced this pull request Jan 19, 2026

chore(pipeline/multiarch): don't rebuild images of architectures alre…

778bc24

…ady built elsewhere (#2155)

lemeurherve added a commit that referenced this pull request Jan 19, 2026

chore(pipeline/multiarch): don't rebuild images of architectures alre…

da0b395

…ady built elsewhere (#2155)

lemeurherve added a commit that referenced this pull request Jan 19, 2026

chore(pipeline/multiarch): don't rebuild images of architectures alre…

e76fe13

…ady built elsewhere (#2155)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

chore(pipeline/multiarch): don't rebuild images of architectures already built elsewhere #2155

chore(pipeline/multiarch): don't rebuild images of architectures already built elsewhere #2155

Uh oh!

lemeurherve commented Dec 24, 2025 •

edited

Loading

Uh oh!

dduportal commented Dec 25, 2025

Uh oh!

Uh oh!

lemeurherve Dec 25, 2025 •

edited

Loading

Uh oh!

Uh oh!

lemeurherve commented Dec 25, 2025 •

edited

Loading

Uh oh!

lemeurherve commented Dec 25, 2025 •

edited

Loading

Uh oh!

Uh oh!

timja Dec 27, 2025

Uh oh!

lemeurherveCB Dec 28, 2025

Uh oh!

timja Dec 28, 2025

Uh oh!

lemeurherveCB Dec 28, 2025 •

edited

Loading

Uh oh!

timja Dec 28, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

chore(pipeline/multiarch): don't rebuild images of architectures already built elsewhere #2155

chore(pipeline/multiarch): don't rebuild images of architectures already built elsewhere #2155

Uh oh!

Conversation

lemeurherve commented Dec 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Testing done

Submitter checklist

Uh oh!

dduportal commented Dec 25, 2025

Uh oh!

Uh oh!

lemeurherve Dec 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lemeurherve commented Dec 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lemeurherve commented Dec 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

timja Dec 27, 2025

Choose a reason for hiding this comment

Uh oh!

lemeurherveCB Dec 28, 2025

Choose a reason for hiding this comment

Uh oh!

timja Dec 28, 2025

Choose a reason for hiding this comment

Uh oh!

lemeurherveCB Dec 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

timja Dec 28, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

lemeurherve commented Dec 24, 2025 •

edited

Loading

lemeurherve Dec 25, 2025 •

edited

Loading

lemeurherve commented Dec 25, 2025 •

edited

Loading

lemeurherve commented Dec 25, 2025 •

edited

Loading

lemeurherveCB Dec 28, 2025 •

edited

Loading