Skip to content

CLOUDP-295785 - using build_info.json in new atomic pipeline #317

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 87 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 86 commits
Commits
Show all changes
87 commits
Select commit Hold shift + click to select a range
6649987
Re-design pipeline
Julien-Ben Jun 12, 2025
675bee4
Remove file
Julien-Ben Jul 29, 2025
833e25f
Put lib back in dependencies
Julien-Ben Jul 29, 2025
15e7f51
add todo
Julien-Ben Jul 29, 2025
120c1af
Fix
Julien-Ben Jul 29, 2025
c9ceabf
Remove multi arch call, fix test image path
Julien-Ben Jul 29, 2025
fb87f4d
Fix agent version for default case
Julien-Ben Jul 30, 2025
c05e180
Lindt
Julien-Ben Jul 30, 2025
747c4ba
isort
Julien-Ben Jul 30, 2025
03fd9b8
Cleanup TODOs
Julien-Ben Jul 30, 2025
1fbb8d5
Rename arch -> platform
Julien-Ben Aug 4, 2025
e9a524f
Don't rely on exception to check for builder existence
Julien-Ben Aug 4, 2025
fa6b899
Remove unused variables
Julien-Ben Aug 4, 2025
426e522
Pre commit
Julien-Ben Aug 4, 2025
6890858
Cleanup
Julien-Ben Aug 4, 2025
aab9592
Correct build envs
Julien-Ben Aug 4, 2025
33173bb
Lindt
Julien-Ben Aug 4, 2025
74e867c
Update Makefile
Julien-Ben Aug 4, 2025
b13b054
Add TODO
Julien-Ben Aug 4, 2025
832ce61
Revert "Pre commit"
Julien-Ben Aug 4, 2025
bc23827
Remove agent unused functions
Julien-Ben Aug 6, 2025
03062c3
Remove debug
Julien-Ben Aug 6, 2025
4d4e4e9
Remove args iteration for multi platform
Julien-Ben Aug 6, 2025
7c9967a
Cleanup unused stuff
Julien-Ben Aug 6, 2025
a7c63c9
Cleanup
Julien-Ben Aug 6, 2025
742e784
Rename file
Julien-Ben Aug 6, 2025
1f0a21b
Remove cli sbom
Julien-Ben Aug 6, 2025
813d539
Renamed image building file
Julien-Ben Aug 6, 2025
c06061b
Freeze python on whales
Julien-Ben Aug 6, 2025
5f9d49a
Lint
Julien-Ben Aug 6, 2025
f390dc9
Remove everything SBOM related
Julien-Ben Aug 6, 2025
a47341d
Lint
Julien-Ben Aug 6, 2025
972b23c
Add TODO
Julien-Ben Aug 6, 2025
4ae4034
Remove --all-agents
Julien-Ben Aug 6, 2025
291f043
Merge branch 'master' into julienben/redesign-pipeline
MaciejKaras Aug 7, 2025
88c76bc
Rename trace
Julien-Ben Aug 7, 2025
0fd4db8
Remove operator build
Julien-Ben Aug 7, 2025
ee86ebf
Doc and logs
Julien-Ben Aug 7, 2025
5f5940f
Use build_image_generic for test images too
Julien-Ben Aug 7, 2025
6dd208f
Remove unused sign images in repositories
Julien-Ben Aug 7, 2025
493d4d6
Remove pipeline_process_image
Julien-Ben Aug 7, 2025
a21b254
Remove process_image
Julien-Ben Aug 7, 2025
a7db180
Rename function
Julien-Ben Aug 7, 2025
52b8662
Lint
Julien-Ben Aug 7, 2025
e465582
wip: Initial changes
MaciejKaras Aug 1, 2025
8a4e4b7
merge redesign-pipeline branch
MaciejKaras Aug 6, 2025
5bfacf6
Fixes after merging remote branch
MaciejKaras Aug 6, 2025
9733b45
Add sign option + change staging registries
MaciejKaras Aug 6, 2025
5ab9c08
Add agent and ops-manager to build_info.json
MaciejKaras Aug 6, 2025
e6d67ca
Fix issue with scenario
MaciejKaras Aug 6, 2025
7a34092
Fix for `build_info` computation
MaciejKaras Aug 7, 2025
a71ccf6
Pipeline fixes
MaciejKaras Aug 7, 2025
3be7731
Remove `all-agents` option
MaciejKaras Aug 7, 2025
1aae28b
Add missing `--sign` option
MaciejKaras Aug 7, 2025
e00b0e3
Fix agent matrix build
MaciejKaras Aug 7, 2025
f894e5b
Disable concurrent builds
MaciejKaras Aug 7, 2025
b140220
Move all env vars to constants.py
MaciejKaras Aug 7, 2025
7b5a064
Remove `operator-quick` image build process
MaciejKaras Aug 7, 2025
8df2ce4
Rebase fix
MaciejKaras Aug 7, 2025
b6b05aa
Add release scenarios for agent and ops-manager
MaciejKaras Aug 7, 2025
c5ad3c5
merge from Julien branch
MaciejKaras Aug 8, 2025
031dd5b
Merge branch 'master' into julienben/redesign-pipeline
Julien-Ben Aug 8, 2025
8183ec5
Fix release_info.py structure
MaciejKaras Aug 8, 2025
e7ab93b
Merge branch 'julienben/redesign-pipeline' into maciejk/use-ar-relate…
MaciejKaras Aug 8, 2025
2ec7587
Explicitly push to ECR with latest tag. Staging as a followup
Julien-Ben Aug 8, 2025
1badff0
Ensure builder in main to fix race conditions
Julien-Ben Aug 8, 2025
9e2815a
Log line
Julien-Ben Aug 8, 2025
e17b323
Remove unused is_running_in_evg_pipeline
Julien-Ben Aug 8, 2025
075fcae
Typo from merge conflict
Julien-Ben Aug 8, 2025
afc9b79
Follow up TODO
Julien-Ben Aug 8, 2025
3ef9e2c
Login for garasign image
Julien-Ben Aug 8, 2025
d2a6153
Handle builder creation race condition with an exception
Julien-Ben Aug 8, 2025
c6fc163
Cleanup ensure
Julien-Ben Aug 8, 2025
347b44b
Revert "Handle builder creation race condition with an exception"
Julien-Ben Aug 8, 2025
e2a4126
Revert "Cleanup ensure"
Julien-Ben Aug 8, 2025
715c4ad
Rename trace
Julien-Ben Aug 8, 2025
e117f79
Merge branch 'julienben/redesign-pipeline' into maciejk/use-ar-relate…
MaciejKaras Aug 8, 2025
2125cb7
Remove comment
Julien-Ben Aug 8, 2025
d047de5
Merge branch 'julienben/redesign-pipeline' into maciejk/use-ar-relate…
MaciejKaras Aug 8, 2025
c8b66fe
Merge branch 'master' into maciejk/use-ar-related-versions
MaciejKaras Aug 8, 2025
fa9c7ee
Review fixes
MaciejKaras Aug 11, 2025
e56591c
Merge branch 'master' into maciejk/use-ar-related-versions
MaciejKaras Aug 11, 2025
597edf2
Added comment to get_version_id() method
MaciejKaras Aug 11, 2025
857f705
Revert parallel `max_workers = 1`
MaciejKaras Aug 11, 2025
59e4420
Agent image fix
MaciejKaras Aug 11, 2025
2eebf0f
Merge branch 'master' into maciejk/use-ar-related-versions
MaciejKaras Aug 11, 2025
bec8626
Merge branch 'master' into maciejk/use-ar-related-versions
MaciejKaras Aug 11, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .evergreen.yml
Original file line number Diff line number Diff line change
Expand Up @@ -423,15 +423,15 @@ tasks:
- func: build_multi_cluster_binary
- func: pipeline
vars:
image_name: test
image_name: meko-tests

- name: build_mco_test_image
commands:
- func: clone
- func: setup_building_host
- func: pipeline
vars:
image_name: mco-test
image_name: mco-tests

- name: build_operator_ubi
commands:
Expand Down
123 changes: 109 additions & 14 deletions build_info.json
Original file line number Diff line number Diff line change
@@ -1,86 +1,94 @@
{
"images": {
"mongodbOperator": {
"operator": {
"patch": {
"repository": "268558157000.dkr.ecr.us-east-1.amazonaws.com/dev/mongodb-kubernetes",
"platforms": [
"linux/amd64"
]
},
"staging": {
"repository": "quay.io/mongodb/mongodb-kubernetes-stg",
"sign": true,
"repository": "268558157000.dkr.ecr.us-east-1.amazonaws.com/staging/mongodb-kubernetes",
"platforms": [
"linux/arm64",
"linux/amd64"
]
},
"release": {
"sign": true,
"repository": "quay.io/mongodb/mongodb-kubernetes",
"platforms": [
"linux/arm64",
"linux/amd64"
]
}
},
"initDatabase": {
"init-database": {
"patch": {
"repository": "268558157000.dkr.ecr.us-east-1.amazonaws.com/dev/mongodb-kubernetes-init-database",
"platforms": [
"linux/amd64"
]
},
"staging": {
"repository": "quay.io/mongodb/mongodb-kubernetes-init-database-stg",
"sign": true,
"repository": "268558157000.dkr.ecr.us-east-1.amazonaws.com/staging/mongodb-kubernetes-init-database",
"platforms": [
"linux/arm64",
"linux/amd64"
]
},
"release": {
"sign": true,
"repository": "quay.io/mongodb/mongodb-kubernetes-init-database",
"platforms": [
"linux/arm64",
"linux/amd64"
]
}
},
"initAppDb": {
"init-appdb": {
"patch": {
"repository": "268558157000.dkr.ecr.us-east-1.amazonaws.com/dev/mongodb-kubernetes-init-appdb",
"platforms": [
"linux/amd64"
]
},
"staging": {
"repository": "quay.io/mongodb/mongodb-kubernetes-init-appdb-stg",
"sign": true,
"repository": "268558157000.dkr.ecr.us-east-1.amazonaws.com/staging/mongodb-kubernetes-init-appdb",
"platforms": [
"linux/arm64",
"linux/amd64"
]
},
"release": {
"sign": true,
"repository": "quay.io/mongodb/mongodb-kubernetes-init-appdb",
"platforms": [
"linux/arm64",
"linux/amd64"
]
}
},
"initOpsManager": {
"init-ops-manager": {
"patch": {
"repository": "268558157000.dkr.ecr.us-east-1.amazonaws.com/dev/mongodb-kubernetes-init-ops-manager",
"platforms": [
"linux/amd64"
]
},
"staging": {
"repository": "quay.io/mongodb/mongodb-kubernetes-init-ops-manager-stg",
"sign": true,
"repository": "268558157000.dkr.ecr.us-east-1.amazonaws.com/staging/mongodb-kubernetes-init-ops-manager",
"platforms": [
"linux/arm64",
"linux/amd64"
]
},
"release": {
"sign": true,
"repository": "quay.io/mongodb/mongodb-kubernetes-init-ops-manager",
"platforms": [
"linux/arm64",
Expand All @@ -96,65 +104,148 @@
]
},
"staging": {
"repository": "quay.io/mongodb/mongodb-kubernetes-database-stg",
"sign": true,
"repository": "268558157000.dkr.ecr.us-east-1.amazonaws.com/staging/mongodb-kubernetes-database",
"platforms": [
"linux/arm64",
"linux/amd64"
]
},
"release": {
"sign": true,
"repository": "quay.io/mongodb/mongodb-kubernetes-database",
"platforms": [
"linux/arm64",
"linux/amd64"
]
}
},
"readinessprobe": {
"meko-tests": {
"patch": {
"repository": "268558157000.dkr.ecr.us-east-1.amazonaws.com/dev/mongodb-kubernetes-tests",
"platforms": [
"linux/amd64"
]
},
"staging": {
"repository": "268558157000.dkr.ecr.us-east-1.amazonaws.com/staging/mongodb-kubernetes-tests",
"platforms": [
"linux/amd64"
]
}
},
"mco-tests": {
"patch": {
"repository": "268558157000.dkr.ecr.us-east-1.amazonaws.com/dev/mongodb-community-tests",
"platforms": [
"linux/amd64"
]
},
"staging": {
"repository": "268558157000.dkr.ecr.us-east-1.amazonaws.com/staging/mongodb-community-tests",
"platforms": [
"linux/amd64"
]
}
},
"readiness-probe": {
"patch": {
"repository": "268558157000.dkr.ecr.us-east-1.amazonaws.com/dev/mongodb-kubernetes-readinessprobe",
"platforms": [
"linux/amd64"
]
},
"staging": {
"repository": "quay.io/mongodb/mongodb-kubernetes-readinessprobe-stg",
"sign": true,
"repository": "268558157000.dkr.ecr.us-east-1.amazonaws.com/staging/mongodb-kubernetes-readinessprobe",
"platforms": [
"linux/arm64",
"linux/amd64"
]
},
"release": {
"version": "1.0.22",
"sign": true,
"repository": "quay.io/mongodb/mongodb-kubernetes-readinessprobe",
"platforms": [
"linux/arm64",
"linux/amd64"
]
}
},
"operator-version-upgrade-post-start-hook": {
"upgrade-hook": {
"patch": {
"repository": "268558157000.dkr.ecr.us-east-1.amazonaws.com/dev/mongodb-kubernetes-operator-version-upgrade-post-start-hook",
"platforms": [
"linux/amd64"
]
},
"staging": {
"repository": "quay.io/mongodb/mongodb-kubernetes-operator-version-upgrade-post-start-hook-stg",
"sign": true,
"repository": "268558157000.dkr.ecr.us-east-1.amazonaws.com/staging/mongodb-kubernetes-operator-version-upgrade-post-start-hook",
"platforms": [
"linux/arm64",
"linux/amd64"
]
},
"release": {
"version": "1.0.9",
"sign": true,
"repository": "quay.io/mongodb/mongodb-kubernetes-operator-version-upgrade-post-start-hook",
"platforms": [
"linux/arm64",
"linux/amd64"
]
}
},
"agent": {
"patch": {
"repository": "268558157000.dkr.ecr.us-east-1.amazonaws.com/dev/mongodb-agent-ubi",
"platforms": [
"linux/amd64"
]
},
"staging": {
"sign": true,
"repository": "268558157000.dkr.ecr.us-east-1.amazonaws.com/staging/mongodb-agent-ubi",
"platforms": [
"linux/arm64",
"linux/amd64"
]
},
"release": {
"sign": true,
"repository": "quay.io/mongodb/mongodb-agent-ubi",
"platforms": [
"linux/arm64",
"linux/amd64"
]
}
},
"ops-manager": {
"patch": {
"version": "om-version-from-release.json",
"repository": "268558157000.dkr.ecr.us-east-1.amazonaws.com/dev/mongodb-enterprise-ops-manager",
"platforms": [
"linux/amd64"
]
},
"staging": {
"version": "om-version-from-release.json",
"sign": true,
"repository": "268558157000.dkr.ecr.us-east-1.amazonaws.com/staging/mongodb-enterprise-ops-manager",
"platforms": [
"linux/amd64"
]
},
"release": {
"version": "om-version-from-release.json",
"sign": true,
"repository": "quay.io/mongodb/mongodb-enterprise-ops-manager",
"platforms": [
"linux/amd64"
]
}
}
},
"binaries": {
Expand All @@ -166,6 +257,7 @@
]
},
"staging": {
"sign": true,
"s3-store": "s3://kubectl-mongodb/staging",
"platforms": [
"darwin/amd64",
Expand All @@ -175,6 +267,7 @@
]
},
"release": {
"sign": true,
"s3-store": "s3://kubectl-mongodb/prod",
"platforms": [
"darwin/amd64",
Expand All @@ -191,9 +284,11 @@
"repository": "268558157000.dkr.ecr.us-east-1.amazonaws.com/dev/helm-charts"
},
"staging": {
"repository": "quay.io/mongodb/helm-charts-stg"
"sign": true,
"repository": "268558157000.dkr.ecr.us-east-1.amazonaws.com/staging/helm-charts"
},
"release": {
"sign": true,
"repository": "quay.io/mongodb/helm-charts"
}
}
Expand Down
4 changes: 1 addition & 3 deletions docker/mongodb-community-tests/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,7 @@
#
# Ref: https://cryptography.io/en/latest/installation/#building-cryptography-on-linux
#
ARG GOLANG_VERSION
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this chagne?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to align with other Dockerfiles, where Golang version is specified in base image


FROM public.ecr.aws/docker/library/golang:${GOLANG_VERSION} as builder
FROM public.ecr.aws/docker/library/golang:1.24 as builder

ENV GO111MODULE=on
ENV GOPATH ""
Expand Down
4 changes: 2 additions & 2 deletions pipeline.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,12 +46,12 @@
get_supported_operator_versions,
get_supported_version_for_image_matrix_handling,
)
from scripts.evergreen.release.images_signing import (
from scripts.evergreen.release.sbom import generate_sbom, generate_sbom_for_cli
from scripts.release.build.image_signing import (
mongodb_artifactory_login,
sign_image,
verify_signature,
)
from scripts.evergreen.release.sbom import generate_sbom, generate_sbom_for_cli

TRACER = trace.get_tracer("evergreen-agent")

Expand Down
2 changes: 1 addition & 1 deletion pipeline_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
is_version_in_range,
operator_build_configuration,
)
from scripts.evergreen.release.images_signing import run_command_with_retries
from scripts.release.build.image_signing import run_command_with_retries

release_json = {
"supportedImages": {
Expand Down
Loading