-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Add bulk tagger #1954
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+162
−0
Merged
Add bulk tagger #1954
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,109 @@ | ||
| #!/usr/bin/env bash | ||
| set -euo pipefail | ||
|
|
||
| trap 'echo "Ctrl+C caught, exiting..."; exit 130' INT | ||
|
|
||
| function exec_tag() { | ||
| local image=$1 | ||
| local tag_prefix=$2 | ||
| local -n hashes=$3 | ||
| local count=0 | ||
| local total="${#hashes[@]}" | ||
| for h in "${hashes[@]}"; do | ||
| local sha256="${h#sha256:}" | ||
| local img_path="${image}@${h}" | ||
| local tag="${tag_prefix}${sha256}" | ||
| echo -n "tagging $img_path with $tag " | ||
| if [[ ${dry_run} == false ]]; then | ||
| gcrane tag "$img_path" "$tag" && echo "SUCCESS $((++count))/$total" || { echo "FAILED"; exit 1; } | ||
| else | ||
| echo "DRY_RUN" | ||
| fi | ||
| done | ||
| } | ||
|
|
||
| function tag_update() { | ||
| local tag_prefix="update-available-" | ||
| local image="$1" | ||
|
|
||
| local NOW | ||
| local images_json | ||
|
|
||
| NOW=$(date +%s000) | ||
| images_json=$(gcrane ls "$image" --json) | ||
|
|
||
| # get all hashes for images that have no tags (or just the commit hash tag) | ||
| # jq behavior for all(test(pattern)) matches the empty list [] | ||
| # or are older than 2 days (we don't want to accidentally mess with any | ||
| # ongoing builds) | ||
| readarray -t targets < <(echo "$images_json" | jq -er --arg now "$NOW" ' | ||
| .manifest | to_entries | sort_by(.value.timeUploadedMs|tonumber) | .[] | select( | ||
| (.value.tag // [] | all(test(".*-[a-f0-9]{40}$|^[a-f0-9]{40}$"))) | ||
loosebazooka marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| and | ||
| (($now | tonumber) - (.value.timeUploadedMs | tonumber) > 172800000) | ||
loosebazooka marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| ) | .key | ||
| '); | ||
loosebazooka marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| echo "tagging ${#targets[@]} images of $image" | ||
|
|
||
| exec_tag "$image" "$tag_prefix" targets | ||
| } | ||
|
|
||
| function tag_deprecate() { | ||
| local targets | ||
| local tag_prefix="deprecated-public-image-" | ||
| local image=$1 | ||
|
|
||
| local images_json | ||
| images_json=$(gcrane ls "$image" --json) | ||
|
|
||
| # get all hashes for all images don't have the deprecated tag | ||
| readarray -t targets < <(echo "$images_json" | jq -er ' | ||
| .manifest | to_entries | sort_by(.value.timeUploadedMs|tonumber) | .[] | select( | ||
| .value.tag // [] | all(test("deprecated-public-image-[a-f0-9]{64}$") | not) | ||
| ) | .key | ||
| '); | ||
|
|
||
| echo "tagging ${#targets[@]} images of $image" | ||
| echo "disabled for now, edit out comment" | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. are you going to leave this commented out?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah I don't have a use for it until Debian 14 |
||
| # exec_tag "$image" "$tag_prefix" targets | ||
| } | ||
|
|
||
| dry_run=${DRY_RUN:-true} | ||
| if [[ "${dry_run}" != "true" && "${dry_run}" != "false" ]]; then | ||
| echo "Error: DRY_RUN needs to have value (true|false)." | ||
| exit 1 | ||
| fi | ||
|
|
||
| function tag() { | ||
| local tag_mode=$1 | ||
| local image=$2 | ||
| case "$tag_mode" in | ||
| update) | ||
| tag_update "$image" | ||
| ;; | ||
| deprecate) | ||
| tag_deprecate "$image" | ||
| ;; | ||
| *) | ||
| echo "wtf?" | ||
loosebazooka marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| exit 1 | ||
loosebazooka marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| esac | ||
| } | ||
loosebazooka marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| function usage() { | ||
| echo "USAGE: $1 [deprecate|update] <image>" >&2 | ||
| exit 1 | ||
| } | ||
loosebazooka marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| case "${1:-"~~nocmd"}" in | ||
| update | deprecate) | ||
| if [[ "$#" -ne 2 ]]; then | ||
| usage "$0" | ||
| fi | ||
| tag "$@" | ||
| ;; | ||
| ~~nocmd | *) | ||
| usage "$0" | ||
| ;; | ||
| esac | ||
loosebazooka marked this conversation as resolved.
Show resolved
Hide resolved
loosebazooka marked this conversation as resolved.
Show resolved
Hide resolved
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| # Container Image Lifecycle Tagger (Distroless Admin Tooling) | ||
|
|
||
| A Bash utility to bulk-tag container images in a registry. This is particularly useful for marking old images as "available for update" or "deprecated" without deleting them, allowing for safer cleanup or auditing. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| * **[gcrane](https://github.com/google/go-containerregistry/tree/main/cmd/gcrane):** Used for interacting with the container registry. | ||
| * **[jq](https://github.com/jqlang/jq/):** Used for parsing registry JSON manifests. | ||
| * **Authentication:** Ensure you are authenticated to your registry (e.g., `gcloud auth configure-docker`). | ||
|
|
||
| ## Functionality | ||
|
|
||
| The script operates in two primary modes: | ||
|
|
||
| ### 1. `update` mode | ||
|
|
||
| Identifies images that are "stale" but likely safe to mark for replacement. It selects images that: | ||
|
|
||
| * Have no tags **OR** only have auto-generated git commit hash based tags (40-character hex strings). | ||
| * Are older than **48 hours** (to avoid interfering with active CI/CD pipelines). | ||
| * **Action:** Tags them with the prefix `update-available-<sha256>`. | ||
|
|
||
| ### 2. `deprecate` mode | ||
|
|
||
| Identifies images that have not yet been marked as deprecated. | ||
|
|
||
| * Selects images that do **not** already have a tag matching the deprecation pattern. | ||
| * **Action:** Tags them with the prefix `deprecated-public-image-<sha256>`. | ||
loosebazooka marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| * *Note: The execution for this mode is currently commented out in the script as a safety measure.* | ||
|
|
||
| --- | ||
|
|
||
| ## Usage | ||
|
|
||
| ```bash | ||
| ./tag.sh [update|deprecate] <image> | ||
| ``` | ||
|
|
||
| ### Examples | ||
|
|
||
| **Dry Run (Default):** | ||
| By default, the script only prints what it *would* do. | ||
|
|
||
| ```bash | ||
| ./tag.sh update gcr.io/distroless/python3-debian12 | ||
| ``` | ||
|
|
||
| **Live Execution:** | ||
| To actually apply tags to the registry, set `DRY_RUN` to `false`. | ||
|
|
||
| ```bash | ||
| DRY_RUN=false ./tag.sh update gcr.io/distroless/python3-debian12 | ||
| ``` | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.