Skip to content

🪞 [MIRRORED] fix: Annotate PLR when when started status is reported #2209

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 6 commits into from

Conversation

zakisk
Copy link
Contributor

@zakisk zakisk commented Aug 7, 2025

🔄 Mirrors #2208 to run E2E tests. Original author: @gbenhaim

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary of Changes

Hello @zakisk, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a mechanism to precisely track and report the "started" status of PipelineRuns to Source Code Management (SCM) providers. Previously, there might have been issues with reporting the "in progress" status multiple times or incorrectly. My changes ensure that the SCM provider is notified exactly once when a PipelineRun transitions from a queued state to a running state, improving the accuracy and reliability of status updates.

Highlights

  • Dedicated Annotation for SCM Reporting: I've introduced a new annotation key, SCMReportingPLRStarted, to explicitly mark when a PipelineRun's "started" status has been reported to the SCM provider. This prevents redundant or incorrect status updates.
  • Refined PipelineRun Status Logic: The logic for updating PipelineRun states and reporting them to SCM providers has been enhanced. Now, the system checks this new annotation before sending an "in progress" status, ensuring it's only sent once per PipelineRun.
  • Improved Reconciliation Robustness: I've added a step in the reconciler to fetch the latest PipelineRun object directly from the API server. This mitigates potential issues caused by stale informer caches, ensuring that reconciliation decisions are always based on the most current state.
  • Enhanced Observability: I've added more detailed logging to provide better insights into the PipelineRun patching process, including the values of the new SCMReportingPLRStarted annotation.
  • Comprehensive Test Coverage: New unit tests have been added to thoroughly validate the behavior of the new SCMReportingPLRStarted annotation and the updated reconciliation logic, ensuring the fix works as expected.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in issue comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments or fill out our survey to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new annotation, SCMReportingPLRStarted, to track whether a PipelineRun's 'started' status has been reported to the SCM. This prevents duplicate status updates and makes the reconciliation logic more robust. The changes are well-implemented across the codebase, including updates to the reconciler, the main pipeline-as-code logic, and corresponding tests.

My review identifies one issue in a new test case where a necessary field was not initialized, which could lead to a panic. I've provided a fix for this.

Comment on lines +435 to +447
r := &Reconciler{
repoLister: informers.Repository.Lister(),
run: &params.Run{
Clients: clients.Clients{
Tekton: stdata.Pipeline,
},
Info: info.Info{
Pac: &info.PacOpts{
Settings: settings.Settings{},
},
},
},
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The Reconciler instance in this test is missing the initialization for the kinteract field. This can lead to a nil pointer dereference when ReconcileKind calls updatePipelineRunToInProgress, which in turn uses kinteract to interact with secrets.

The run.Clients.Kube field also needs to be initialized.

Please update the Reconciler initialization to include kinteract and the Kubernetes client, like this:

			r := &Reconciler{
				repoLister: informers.Repository.Lister(),
				run: &params.Run{
					Clients: clients.Clients{
						Tekton: stdata.Pipeline,
						Kube:   stdata.Kube,
					},
					Info: info.Info{
						Pac: &info.PacOpts{
							Settings: settings.Settings{},
						},
					},
				},
				kinteract: &kubeinteraction.Interaction{
					Run: &params.Run{
						Clients: clients.Clients{
							Kube:   stdata.Kube,
							Tekton: stdata.Pipeline,
						},
					},
				},
			}

This will make the test more robust and prevent potential panics.

The use of the state label (which is mutable) for deciding when to report
to the SCM that the PLR was started is flaky.

It was seen that the reconciler get events about PLRs with unexpected
value for the state label. For example, after the status is reported
to the SCM, and the label value is patched to "started", after serval
reconcile iterations the label had the "queued" value again. This can
happen because of unsafe patching done by controllers
(not just the PAC controllers) which reconciles PLRs.

Introduce a new annotation for indicating the the status was reported
to the SCM. By adding an annotation which is set once, we remove the
risk that its value will get overwritten by other controllers
(since maps are merged when patched, values are not getting removed
unless explicitly defined in the patch
- https://datatracker.ietf.org/doc/html/rfc7386#section-2)

In addition, at the start of each reconcile loop, ensure that we
operate on the latest version of the PLR and not using a stale value
from the cache.

Assisted-By: Cursor
Signed-off-by: Gal Ben Haim <[email protected]>
Set the state annotation and labels in pipelineascode only after
the state was reported to the SCM.

This is achieved by patching the PLR with the state label/annotation
after it was created.

This change is needed in order to avoid a case where the watcher will report
a status before PAC.

With this change the issue can't happen, since the watcher only reconciles
PLRs that has the state annotation, so the first reconcile of the PLRs
will happen only after PAC already sent the initial status to the SCM.

In addition, simplify the condition which checks if the "queued" state
annotation/label should be set. We can only check if the PLR is pending,
it doesn't matter if the reason is because of PAC's concurrency control
or an external controller.

Signed-off-by: Gal Ben Haim <[email protected]>
@zakisk zakisk force-pushed the test-pr-2208-gbenhaim branch from 56793cb to 25bfed7 Compare August 7, 2025 07:55
gbenhaim and others added 4 commits August 7, 2025 15:43
Instead of running the reconcile loop on the fresh pipeline, skip it
and let the next iterations to reconcile it.

Signed-off-by: Gal Ben Haim <[email protected]>
@chmouel chmouel closed this Aug 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

3 participants