Promotion 2026-01-06 anvilprod (#7678)

[achave11-ucsc]

Linked issue: #7678

Notes

• In order to minimize the time frame during which the service returns 500 responses, cancel the deploy stage and then be present when starting it manually, cancel the integration_test stage and immediately trigger the early_reindex when the deploy_browser stage finishes

Checklist

Author

• PR is assigned to the author
• Status of PR is In progress
• Target branch is anvilprod
• Name of PR branch matches promotions/yyyy-mm-dd-anvilprod
• PR is linked to the promotion issue it resolves
• Status of linked issue is In progress
• PR description links to linked issue
• Title of linked issue matches Promotion yyyy-mm-dd
• PR title starts with title of linked issue followed by anvilprod
• PR title references the linked issue

Author (reindex)

• This PR is labeled reindex:anvilprod _{or the changes introduced by it will not require reindexing of anvilprod}
• This PR is labeled reindex:partial and its description documents the specific reindexing procedure for anvilprod _{or requires a full reindex or is not labeledreindex:anvilprod}

Author (upgrading deployments)

• This PR is labeled upgrade _{or does not require upgrading deployments}
• This PR is labeled deploy:shared _{or does not modify docker_images.json, and does not require deploying the shared component for any other reason}
• This PR is labeled deploy:gitlab _{or does not require deploying the gitlab component}
• This PR is labeled deploy:runner _{or does not require deploying the runner image}

Author (before every review)

• PR branch is up to date (if not, merge anvilprod into PR branch to integrate upstream changes)
• PR is not a draft
• PR is awaiting requested review from system administrator
• Status of PR is Review requested
• PR is assigned to only the system administrator and the author

System administrator (after approval)

• Actually approved the PR
• Decided if PR can be labeled no sandbox
• N reviews label is accurate
• Status of PR is Approved
• PR is assigned to only the operator and the author

Operator

• Pushed PR branch to GitHub

Operator (deploy .shared and .gitlab components)

• Ran _select anvilprod.shared && CI_COMMIT_REF_NAME=anvilprod make -C terraform/shared apply_keep_unused _{or this PR is not labeled deploy:shared}
• Ran _select anvilprod.gitlab && python scripts/create_gitlab_snapshot.py --no-restart (see operator manual for details) _{or this PR is not labeled backup:gitlab}
• Ran _select anvilprod.gitlab && CI_COMMIT_REF_NAME=anvilprod make -C terraform/gitlab apply _{or this PR is not labeled deploy:gitlab}
• Checked the items in the next section _{or this PR is labeled deploy:gitlab}
• PR is assigned to only the system administrator and the author _{or this PR is not labeled deploy:gitlab}

System administrator (post-deploy of .gitlab component)

• Background migrations for anvilprod.gitlab are complete _{or this PR is not labeled deploy:gitlab}
• PR is assigned to only the operator and the author

Operator (deploy runner image)

• Ran _select anvilprod.gitlab && make -C terraform/gitlab/runner _{or this PR is not labeled deploy:runner}

Operator (sandbox build)

• Added sandbox label _{or PR is labeled no sandbox}
• Pushed PR branch to GitLab anvilprod _{or PR is labeled no sandbox}
• Build passes in hammerbox deployment _{or PR is labeled no sandbox}
• Reviewed build logs for anomalies in hammerbox deployment _{or PR is labeled no sandbox}
• Deleted unreferenced indices in hammerbox _{or this PR does not remove catalogs or otherwise causes unreferenced indices in hammerbox}
• Started reindex in hammerbox _{or this PR is not labeled reindex:anvilprod}
• Checked for failures in hammerbox _{or this PR is not labeled reindex:anvilprod}

Operator (merge the branch)

• All status checks passed and the PR is mergeable
• The title of the merge commit starts with the title of this PR
• Added PR # reference to merge commit title
• Collected commit title tags in merge commit title _{but excluded any p tags}
• Pushed merge commit to GitHub
• Status of PR is Merged stable

Operator (main build)

• Pushed merge commit to GitLab anvilprod
• Build passes on GitLab anvilprod
• Reviewed build logs for anomalies on GitLab anvilprod
• Ran _select anvilprod.shared && make -C terraform/shared apply _{or this PR is not labeled deploy:shared}
• Deleted PR branch from GitHub
• PR is assigned to only the operator
• Deleted PR branch from GitLab anvilprod
• Status of linked issue is Stable
• Status of promoted¹ PRs is Merged stable
• Status of promoted¹ issues is Stable

¹ Promoted issues and PRs are referenced in the titles of the commits
that the promotion branch introduces to the stable branch. Prior to the
promotion, the status of promoted issues (PRs) is Lower (Merged lower).
Promoted PRs in status Done do not need to be moved.

Operator (reindex)

• Deindexed all unreferenced catalogs in anvilprod _{or this PR is neither labeled reindex:partial nor reindex:anvilprod}
• Deindexed specific sources in anvilprod _{or this PR is neither labeled reindex:partial nor reindex:anvilprod}
• Indexed specific sources in anvilprod _{or this PR is neither labeled reindex:partial nor reindex:anvilprod}
• Started reindex in anvilprod _{or this PR does not require reindexing anvilprod}
• Checked for, triaged and possibly requeued messages in both fail queues in anvilprod _{or this PR does not require reindexing anvilprod}
• Emptied fail queues in anvilprod _{or this PR does not require reindexing anvilprod}
• Restarted the Data Browser pipeline for the ucsc/anvil/anvilprod branch on GitLab in anvilprod _{or this PR does not require reindexing anvilprod}
• Restarted deploy_browser job in the GitLab pipeline for this PR in anvilprod _{or this PR does not require reindexing anvilprod}

Operator (mirroring)

• Started mirroring in anvilprod _{or this PR does not require mirroring anvilprod}
• Checked for, triaged and possibly requeued messages in mirror fail queue in anvilprod _{or this PR does not require mirroring anvilprod}
• Emptied mirror fail queue in anvilprod _{or this PR does not require mirroring anvilprod}

Operator

• PR is assigned to only the system administrator

System administrator

• Removed unused image tags from pycharm image on DockerHub _{or this promotion does not alter references to that image}
• Removed unused image tags from bigquery_emulator image on DockerHub _{or this promotion does not alter references to that image}
• PR is assigned to no one

Shorthand for review comments

• L line is too long
• W line wrapping is wrong
• Q bad quotes
• F other formatting problem

[coveralls]

coverage: 85.043% (-0.01%) from 85.057%
when pulling 4ef3ad2 on promotions/2026-01-06-anvilprod
into db4c8fc on anvilprod.

[achave11-ucsc]

While this API updated doesn't affect the AnVIL prod metadata schema, it still bumps the service API version.