Skip to content

WIP: fix (probable) race condition in promotion #1531

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

justinsb
Copy link
Contributor

@justinsb justinsb commented May 9, 2025

I believe there is a race condition here, and we are closing the channel
during promotion before we finish filling it, in ExecRequests.

This manifests as images not being promoted, seemingly at random,
without errors.

Refactor to make the logic (hopefully) clearer.


Simplify code to address potential race condition that was (likely) causing incomplete image promotion.

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels May 9, 2025
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: justinsb
Once this PR has been reviewed and has the lgtm label, please assign puerco for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot requested a review from jimangel May 9, 2025 01:38
@k8s-ci-robot k8s-ci-robot added the area/artifacts Issues or PRs related to the hosting of release artifacts for subprojects label May 9, 2025
@k8s-ci-robot k8s-ci-robot requested a review from justaugustus May 9, 2025 01:38
@k8s-ci-robot k8s-ci-robot added area/release-eng Issues or PRs related to the Release Engineering subproject sig/release Categorizes an issue or PR as relevant to SIG Release. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels May 9, 2025
@justinsb justinsb force-pushed the simplified_fork_join branch from 45eec75 to 8e8e95f Compare May 9, 2025 01:43

if len(toPromote) == 0 {
logrus.Info("Nothing to promote.")
} else {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not the most idiomatic, but preserves indentation for easier review.

@justinsb justinsb force-pushed the simplified_fork_join branch from 8e8e95f to 2f47ec3 Compare May 9, 2025 01:45
I believe there is a race condition here, and we are closing the channel
during promotion before we finish filling it in ExecRequests.

This manifests as images not being promoted, seemingly at random,
without errors.

Refactor to make the logic (hopefully) clearer.
@justinsb justinsb force-pushed the simplified_fork_join branch from 2f47ec3 to 9cd2dbf Compare May 9, 2025 01:48
@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels May 9, 2025
@justinsb justinsb changed the title WIP: fix race condition in promotion WIP: fix (probable) race condition in promotion May 9, 2025
@k8s-ci-robot
Copy link
Contributor

@justinsb: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-cip-verify 9cd2dbf link true /test pull-cip-verify

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@k8s-ci-robot
Copy link
Contributor

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/artifacts Issues or PRs related to the hosting of release artifacts for subprojects area/release-eng Issues or PRs related to the Release Engineering subproject cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/release Categorizes an issue or PR as relevant to SIG Release. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants