CLOUDP-339878 - add docker caching #360

nammn · 2025-08-21T14:23:58Z

Summary

This pull request introduces a new branch and cache scoping system for Docker BuildKit remote cache in CI and local development environments. It adds robust branch detection and cache scope generation logic, integrates these into the image build process, and provides comprehensive unit tests to ensure correctness. The main goal is to enable per-branch and per-patch caching for Docker builds, improving build performance and cache isolation.

Branch and cache scoping infrastructure:

Added scripts/release/branch_detection.py with functions to detect the current git branch and generate sanitized cache scope strings for use in BuildKit caching, supporting both local development and Evergreen CI environments.

Integration with image build process:

Refactored scripts/release/build/image_build_process.py to use the new branch and cache scope utilities for configuring BuildKit cache, including logic to set up per-branch/per-patch cache repositories and read/write precedence (branch → master).
Ensured that ECR cache repositories are created as needed before builds, and added detailed logging for cache configuration during builds.

Proof of Work

green ci
example output

[2025/08/21 17:03:36.058] #31 exporting cache to registry
[2025/08/21 17:03:36.058] #31 preparing build cache for export
[2025/08/21 17:03:56.826] #31 writing layer sha256:104f1be5715fef257f90b38fddaeee7ef6ae9f1da7e6b03051fdd5ba50f18026
[2025/08/21 17:03:57.846] #31 writing layer sha256:104f1be5715fef257f90b38fddaeee7ef6ae9f1da7e6b03051fdd5ba50f18026 1.0s done
[2025/08/21 17:03:57.846] #31 writing layer sha256:11df871e90940bf39eb8c800c8966b01a29a704e1dc0cdbcd85107e2bbf98948
[2025/08/21 17:03:58.294] #31 writing layer sha256:11df871e90940bf39eb8c800c8966b01a29a704e1dc0cdbcd85107e2bbf98948 0.4s done
[2025/08/21 17:03:58.294] #31 writing layer sha256:14b1babc8f267ede2a5e969d069d4be8f28ada9ff5d950645d8dbed508b8e91d

unit tests

Checklist

Have you linked a jira ticket and/or is the ticket in the title?
Have you checked whether your jira ticket required DOCSP changes?
Have you added changelog file?
- use skip-changelog label if not needed
- refer to Changelog files and Release Notes section in CONTRIBUTING.md for more details

github-actions · 2025-08-21T14:24:41Z

⚠️ (this preview might not be accurate if the PR is not rebased on current master branch)

MCK 1.3.0 Release Notes

New Features

Multi-Architecture Support

We've added comprehensive multi-architecture support for the kubernetes operator. This enhancement enables deployment on IBM Power (ppc64le) and IBM Z (s390x) architectures alongside
existing x86_64 support. Core images (operator, agent, init containers, database, readiness probe) now support multiple architectures. We do not add support IBM and ARM support for Ops-Manager and the init-ops-manager image.

MongoDB Agent images have been migrated to new container repository: quay.io/mongodb/mongodb-agent.
- the agents in the new repository will support the x86-64, ARM64, s390x, and ppc64le architectures. More can be read in the public docs.
- operator running >=MCK1.3.0 and static cannot use the agent images from the old container repository quay.io/mongodb/mongodb-agent-ubi.
quay.io/mongodb/mongodb-agent-ubi should not be used anymore, it's only there for backwards compatibility.

Bug Fixes

This change fixes the current complex and difficult-to-maintain architecture for stateful set containers, which relies on an "agent matrix" to map operator and agent versions which led to a sheer amount of images.
We solve this by shifting to a 3-container setup. This new design eliminates the need for the operator-version/agent-version matrix by adding one additional container containing all required binaries. This architecture maps to what we already do with the mongodb-database container.
Fixed an issue where the readiness probe reported the node as ready even when its authentication mechanism was not in sync with the other nodes, potentially causing premature restarts.