Skip to content

Commit 696a572

Browse files
committed
jobs/build: don't try to complete ongoing builds
Since 973bcf9 ("jobs/build: rerun `build-arch` if previous build is incomplete"), there is a race possible where the `build` job rerun logic could kick in before the `release` jobs initially triggered for that build has finished. We don't want to queue builds in that case. Gate the rerun logic on whether the multi-arch locks and release lock are taken. It's theoretically possible but highly unlikely that we probe the lock status before the previous `release` job takes it. Ideally, we would have a way to directly take the lock and "transfer" its ownership to the job we trigger. Anyway, if that somehow happens, it would result in the job being run twice, which is safe.
1 parent c2ad35d commit 696a572

File tree

1 file changed

+15
-3
lines changed

1 file changed

+15
-3
lines changed

jobs/build.Jenkinsfile

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -265,9 +265,10 @@ lock(resource: "build-${params.STREAM}") {
265265
// Nothing changed since the latest build. Check if it's missing
266266
// some arches and retrigger `build-arch` only for the missing
267267
// arches, and the follow-up `release` job. Match the exact src
268-
// config commit that was used. Skip if not uploading since it's
269-
// required for multi-arch.
270-
if (uploading) {
268+
// config commit that was used. But only do this if there isn't
269+
// already outstanding work in progress for that build ID. Skip if
270+
// not uploading since it's required for multi-arch.
271+
if (uploading && !buildid_has_work_pending(buildID, additional_arches)) {
271272
def builds = readJSON file: "builds/builds.json"
272273
assert buildID == builds.builds[0].id
273274
def missing_arches = additional_arches - builds.builds[0].arches
@@ -542,3 +543,14 @@ def run_release_job(buildID) {
542543
]
543544
}
544545
}
546+
547+
def buildid_has_work_pending(buildID, arches) {
548+
def locked = true
549+
// these locks match the ones in the release job
550+
def locks = arches.collect{[resource: "release-${buildID}-${it}"]}
551+
lock(resource: "release-${params.STREAM}", extra: locks, skipIfLocked: true) {
552+
// NB: `return` here wouldn't actually return from the function
553+
locked = false
554+
}
555+
return locked
556+
}

0 commit comments

Comments
 (0)