-
-
Notifications
You must be signed in to change notification settings - Fork 731
COMP: Backport CI infrastructure (ccache, GHA cache, pixi CI tasks, Azure/macOS-arm caching) to release-5.4 #6334
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
hjmjohnson
wants to merge
5
commits into
release-5.4
Choose a base branch
from
ci-backport-rel54
base: release-5.4
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
4fdb4e5
COMP: Add ccache support and pixi CI tasks to release-5.4
hjmjohnson 8308c69
COMP: Backport GitHub Actions cache-management workflows to release-5.4
hjmjohnson bd73d04
COMP: Use pixi CI tasks and cache ccache in release-5.4 pixi workflow
hjmjohnson ee682ce
COMP: Add ccache caching to release-5.4 Azure pipelines
hjmjohnson 25dbd45
COMP: Add ccache caching to release-5.4 macOS-arm workflow
hjmjohnson File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| name: Cleanup PR caches | ||
|
|
||
| # When a pull request closes (merged or not), delete every GitHub Actions | ||
| # cache entry scoped to its merge ref. This reclaims the repo's 10 GB | ||
| # cache budget within seconds of close, without adding any overhead to | ||
| # the regular build/test runs. | ||
| # | ||
| # Reference: | ||
| # https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#force-deleting-cache-entries | ||
|
|
||
| on: | ||
| # Use pull_request_target rather than pull_request: GITHUB_TOKEN is | ||
| # read-only for pull_request events from forks, which would cause every | ||
| # real-world PR cache delete to return 403. pull_request_target runs | ||
| # in the base-repo context with the permissions this workflow requests, | ||
| # and is safe here because the workflow never checks out or executes | ||
| # fork-authored code -- it only calls the GitHub API. | ||
| # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token | ||
| pull_request_target: | ||
| types: [closed] | ||
|
|
||
| jobs: | ||
| purge-pr-caches: | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| actions: write # required to delete caches | ||
| contents: read | ||
| steps: | ||
| - name: Delete all caches for the closed PR | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| REPO: ${{ github.repository }} | ||
| PR_NUM: ${{ github.event.pull_request.number }} | ||
| run: | | ||
| set -euo pipefail | ||
| # Sweep both the merge-ref and the head-ref. Most pull_request | ||
| # workflows cache against refs/pull/N/merge (github.sha resolves | ||
| # to the merge commit), but workflows that key on github.head_ref | ||
| # can write caches to refs/pull/N/head. Cleaning up both is | ||
| # belt-and-braces against future workflow additions. | ||
| total=0 | ||
| for scope in merge head; do | ||
| ref="refs/pull/${PR_NUM}/${scope}" | ||
| echo "Purging caches for ${REPO} at ref=${ref}" | ||
| while read -r id; do | ||
| [ -z "$id" ] && continue | ||
| if gh api -X DELETE "repos/${REPO}/actions/caches/${id}" >/dev/null 2>&1; then | ||
| total=$((total + 1)) | ||
| echo " deleted cache id=${id} (ref=${ref})" | ||
| else | ||
| echo " WARN: failed to delete cache id=${id} (ref=${ref})" | ||
| fi | ||
| done < <(gh api --paginate \ | ||
| "repos/${REPO}/actions/caches?ref=${ref}&per_page=100" \ | ||
| --jq '.actions_caches[].id') | ||
| done | ||
| echo "Purged ${total} cache entries across merge+head refs." |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,84 @@ | ||
| name: Cleanup stale caches (nightly sweep) | ||
|
|
||
| # Safety net for the cleanup-on-close workflow: once per day, scan the | ||
| # repository's GitHub Actions caches and purge any cache scoped to a | ||
| # pull-request merge ref whose PR has been closed for more than a | ||
| # 3-day grace period. The grace period lets anyone spot-rerun a | ||
| # just-merged PR before its caches vanish. | ||
| # | ||
| # This catches the edge cases the pull_request:closed trigger misses: | ||
| # - PRs closed during a cleanup-workflow outage | ||
| # - caches orphaned when a PR was closed before this workflow existed | ||
| # - caches stuck on refs/pull/N/merge after branch deletion | ||
| # | ||
| # Reference: | ||
| # https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#force-deleting-cache-entries | ||
|
|
||
| on: | ||
| schedule: | ||
| - cron: '0 6 * * *' # 06:00 UTC daily | ||
| workflow_dispatch: | ||
|
|
||
| jobs: | ||
| sweep: | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| actions: write # required to delete caches | ||
| pull-requests: read # required to check PR state | ||
| steps: | ||
| - name: Purge caches for PRs closed more than 3 days ago | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| REPO: ${{ github.repository }} | ||
| GRACE_DAYS: '3' | ||
| run: | | ||
| set -euo pipefail | ||
| CUTOFF=$(date -u -d "${GRACE_DAYS} days ago" +%s) | ||
| echo "Grace cutoff: PRs closed before $(date -u -d "@${CUTOFF}" --iso-8601=seconds)" | ||
|
|
||
| # Step 1: enumerate every PR-scoped cache (id + pr number). | ||
| tmpdir=$(mktemp -d) | ||
| trap 'rm -rf "$tmpdir"' EXIT | ||
| gh api --paginate \ | ||
| "repos/${REPO}/actions/caches?per_page=100" \ | ||
| --jq '.actions_caches[] | | ||
| select(.ref | startswith("refs/pull/")) | | ||
| [.id, (.ref | capture("refs/pull/(?<n>[0-9]+)/").n)] | | ||
| @tsv' > "${tmpdir}/caches.tsv" | ||
| total_scanned=$(wc -l < "${tmpdir}/caches.tsv") | ||
|
|
||
| # Step 2: one API call per *distinct* PR (not per cache). | ||
| awk '{print $2}' "${tmpdir}/caches.tsv" | sort -u > "${tmpdir}/prs.txt" | ||
| : > "${tmpdir}/prstate.tsv" | ||
| while read -r pr; do | ||
| info=$(gh pr view "$pr" --repo "$REPO" \ | ||
| --json state,closedAt 2>/dev/null || echo '{}') | ||
| state=$(echo "$info" | jq -r '.state // "UNKNOWN"') | ||
| closed=$(echo "$info" | jq -r '.closedAt // "null"') | ||
| printf '%s\t%s\t%s\n' "$pr" "$state" "$closed" >> "${tmpdir}/prstate.tsv" | ||
| done < "${tmpdir}/prs.txt" | ||
|
|
||
| # Step 3: join caches with PR state and purge those past the grace cutoff. | ||
| total_purged=0 | ||
| while read -r id pr; do | ||
| [ -z "$id" ] && continue | ||
| row=$(awk -v p="$pr" '$1 == p' "${tmpdir}/prstate.tsv") | ||
| state=$(echo "$row" | cut -f2) | ||
| closed=$(echo "$row" | cut -f3) | ||
| if [ "$state" = "OPEN" ] || [ "$closed" = "null" ]; then | ||
| continue | ||
| fi | ||
| closed_ts=$(date -u -d "$closed" +%s 2>/dev/null || echo 0) | ||
| [ "$closed_ts" -eq 0 ] && continue | ||
| if [ "$closed_ts" -lt "$CUTOFF" ]; then | ||
| if gh api -X DELETE "repos/${REPO}/actions/caches/${id}" >/dev/null 2>&1; then | ||
| total_purged=$((total_purged + 1)) | ||
| echo " purged cache id=${id} (PR #${pr} ${state} since ${closed})" | ||
| else | ||
| echo " WARN: failed to delete cache id=${id} (PR #${pr})" | ||
| fi | ||
| fi | ||
| done < "${tmpdir}/caches.tsv" | ||
|
|
||
| distinct_prs=$(wc -l < "${tmpdir}/prs.txt") | ||
| echo "Scanned ${total_scanned} PR-scoped caches across ${distinct_prs} distinct PRs; purged ${total_purged}." |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,102 @@ | ||
| name: Populate ExternalData Cache | ||
|
|
||
| # Single owner of the shared "externaldata-v1-<hashFiles>" GitHub Actions | ||
| # cache entry. Every other workflow restores this entry but never saves | ||
| # it — see the comments in arm.yml and pixi.yml for the race that a | ||
| # multi-writer design caused. | ||
| # | ||
| # The job prefetches every .cid referenced in the source tree through | ||
| # the same gateway list CMake/ITKExternalData.cmake uses, verifies that | ||
| # all objects landed on disk, and only then saves the cache. If any | ||
| # object is missing the save is skipped so a later run can try again | ||
| # without poisoning the key. | ||
|
|
||
| on: | ||
| # PRs that add or modify .cid references produce a new hashFiles | ||
| # digest, so the cache needs to be repopulated for that digest. | ||
| pull_request: | ||
| paths: | ||
| - '**/*.cid' | ||
| # Keep main and release branches' caches populated as new .cid files | ||
| # land. | ||
| push: | ||
| branches: | ||
| - main | ||
| - 'release*' | ||
| paths: | ||
| - '**/*.cid' | ||
| # Nightly safety net: if a populate run was skipped because some CIDs | ||
| # were unreachable on one day, the next night retries. | ||
| schedule: | ||
| - cron: '17 5 * * *' | ||
| workflow_dispatch: | ||
|
|
||
| concurrency: | ||
| # Only one populate job per hashFiles digest makes sense, but we key | ||
| # the concurrency group on the branch ref since hashFiles requires a | ||
| # checkout. Mid-flight runs cancel; the final one wins. | ||
| group: 'externaldata-populate@${{ github.head_ref || github.ref }}' | ||
| cancel-in-progress: true | ||
|
|
||
| permissions: | ||
| contents: read | ||
| actions: write # needed to manage cache entries | ||
|
|
||
| jobs: | ||
| populate: | ||
| name: Populate shared ExternalData cache | ||
| runs-on: ubuntu-22.04 | ||
| timeout-minutes: 60 | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v5 | ||
| with: | ||
| fetch-depth: 1 | ||
|
|
||
| - name: Restore ExternalData object store | ||
| id: restore-externaldata | ||
| uses: actions/cache/restore@v5 | ||
| with: | ||
| path: ${{ runner.temp }}/ExternalData | ||
| key: externaldata-v1-${{ hashFiles('**/*.cid') }} | ||
|
|
||
| - name: Skip if cache already complete | ||
| if: steps.restore-externaldata.outputs.cache-hit == 'true' | ||
| run: echo "Cache already present for this hashFiles digest - nothing to do." | ||
|
|
||
| - name: Prefetch every CID | ||
| if: steps.restore-externaldata.outputs.cache-hit != 'true' | ||
| shell: bash | ||
| env: | ||
| EXTERNALDATA_STORE: ${{ runner.temp }}/ExternalData | ||
| run: | | ||
| python3 Utilities/Maintenance/PrefetchCIDContentLinks.py \ | ||
| --repo-root . \ | ||
| --store "$EXTERNALDATA_STORE" | ||
|
|
||
| # Integrity gate: refuse to save unless every unique CID in the | ||
| # source tree has an object on disk. A partial save under the | ||
| # shared key would propagate holes to every consumer workflow. | ||
| - name: Verify completeness | ||
| if: steps.restore-externaldata.outputs.cache-hit != 'true' | ||
| shell: bash | ||
| env: | ||
| EXTERNALDATA_STORE: ${{ runner.temp }}/ExternalData | ||
| run: | | ||
| expected=$(find . -name '*.cid' -not -path './.git/*' -print0 \ | ||
| | xargs -0 -I{} cat {} \ | ||
| | sort -u | wc -l | tr -d ' ') | ||
| present=$(find "$EXTERNALDATA_STORE/cid" -type f 2>/dev/null | wc -l | tr -d ' ') | ||
| echo "expected unique CIDs: $expected" | ||
| echo "present on disk : $present" | ||
| if [ "$present" -lt "$expected" ]; then | ||
| echo "::error::ExternalData prefetch produced $present/$expected objects; refusing to save a partial cache." | ||
| exit 1 | ||
| fi | ||
|
|
||
| - name: Save ExternalData object store | ||
| if: steps.restore-externaldata.outputs.cache-hit != 'true' | ||
| uses: actions/cache/save@v5 | ||
| with: | ||
| path: ${{ runner.temp }}/ExternalData | ||
| key: externaldata-v1-${{ hashFiles('**/*.cid') }} |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.