⚠️ Migration to the new events API #3262

clebs · 2025-07-22T20:32:25Z

This Pull Request migrates from the old events API (k8s.io/client-go/tools/record) to the new API (k8s.io/client-go/tools/events).

Closes #2141

k8s-ci-robot · 2025-07-22T20:32:35Z

Hi @clebs. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

clebs · 2025-07-22T20:36:47Z

Open Items:

StartRecordingToSink: manage the context/channel in the provider on ensureRecording and Stop.
StartWatcher: see if this is still needed and how to adapt it if it is.
Leader election: resourcelock.ResourceLockConfig is still using the old API.
This has now been solved by adding an adapter that turns old API calls into new API calls (albeit pretty simplistic). So that way on the backend we have only the new API and places that require the old can use the adapter.
Update unit tests.

alvaroaleman · 2025-07-22T20:50:59Z

/ok-to-test

alvaroaleman

two questions

pkg/recorder/recorder.go

alvaroaleman · 2025-07-23T20:40:09Z

pkg/recorder/recorder.go

+	GetEventRecorder(name string) events.EventRecorder
+	// GetOldEventRecorder returns an EventRecorder for the old events API.
+	// The old API is not 100% supported anymore, use the new one whenever possible.
+	GetOldEventRecorder(name string) record.EventRecorder


Rather than extending the interface here, would it be possible to build a wrapper that makes a record.EventRecorder out of a events.EventRecorder?

Yes, it is already built as a wrapper under the hood in intrec. I added it here with the idea to support both APIs for some time. If it is not what we are going for, having the wrapper only where it is needed is better.

I think there are two main ways we could go about this:

We make this a non-breaking change by keeping the existing GetEventRecorderFor with the existing signature and add a new GetEventRecorder with the new events.EventRecorder as return but mark the GetEventRecorderFor as // deprecated. At some point in a future release we will then remove GetEventRecorderFor

We make this a breaking change by changing the signature of the existing GetEventRecorderFor to return an events.EventRecorder and we provide an adapter somewhere to go from events.EventRecorder to a record.EventRecorder for places like the leader election

My vote if this is possible would be to do 1) as this is the least disruptive for c-r users but its possible I've missed something and it isn't possible

Option 1 is doable, but I have to see if the wrapper approach is good enough. Reason being, that the wrapper calls the new API under the hood and both APIs are quite different. The old API has AnnotatedEventf while the new one does not and the new one has things like relatedObject and action.

Otherwise, the other way to support both is having 2 distinct implementations in separate packages and duplicate the broadcaster, provider, interfaces and injection at the manager and cluster levels.
Finally also duplicate the tests so both versions are covered.

I will test if the wrapper is viable, present the results and then we can make an informed decision.

I have tested out the wrapper and it has the one downside: we loose the ability to set annotations on events since there is no equivalent AnnotatedEventf. I also have tried to get some info on why that is the case.

Seeing that, the wrapper can not provide the same features as the old API has, so the only option to support both is probably having both implementations at the same time.

@alvaroaleman my proposal is to duplicate the internal events package and have both implementations separate from each other even if that means lots of duplication for some time. It will make it easier later on to cleanup the old API when removed. What do you think?

@alvaroaleman my proposal is to duplicate the internal events package and have both implementations separate from each other even if that means lots of duplication for some time. It will make it easier later on to cleanup the old API when removed. What do you think?

Not great, but given its for a limited time only that seems okay to me

I am also not a great fan.

I will do the exercise and see how much I can reuse without having a tangled nightmare.

Worst case it is all duplicate.

@alvaroaleman I have gone through this and now this PR supports both the old and new events APIs with as little duplication as possible.

There is still a fair amount of duplication but it is confined mostly to internal code and should still be relatively easy to cleanup later on.

If you agree with this approach I will move to adding tests for the new API and move this out of draft.

If you agree with this approach I will move to adding tests for the new API and move this out of draft.

Sounds good to me

Tests added and moved out of draft. PTAL!
Still finishing up the recorder integration test for the new API, but the code should be ready for review.

pkg/internal/recorder/recorder.go

sbueringer · 2025-08-07T16:39:10Z

/assign

k8s-ci-robot · 2025-08-11T23:02:04Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: clebs
Once this PR has been reviewed and has the lgtm label, please ask for approval from sbueringer. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

pkg/internal/recorder/recorder.go

sbueringer · 2025-08-12T12:10:26Z

pkg/leaderelection/leader_election.go

@@ -128,7 +128,7 @@ func NewResourceLock(config *rest.Config, recorderProvider recorder.Provider, op
 		coordinationClient,
 		resourcelock.ResourceLockConfig{
 			Identity:      id,
-			EventRecorder: recorderProvider.GetEventRecorderFor(id),
+			EventRecorder: recorderProvider.GetEventRecorderFor(id), //nolint:staticcheck


What would be the plan to resolve this one?
Improve the upstream package so it can handle the new recorder, then use it here?

(if yes, let's make sure we have corresponding follow-up issues in CR & k/k)

Correct, this would require client-go to migrate to the new API too. I will make sure to create followups once this is in.

Perfect, thx!

pkg/manager/manager.go

pkg/recorder/recorder.go

sbueringer · 2025-08-12T12:23:53Z

@clebs Just a heads up. If we want to get this into the next CR minor release the deadline is in 2-3 weeks. We want to release the next CR minor shortly after the Kubernetes minor which is scheduled for 27th August

clebs · 2025-08-12T13:20:46Z

@clebs Just a heads up. If we want to get this into the next CR minor release the deadline is in 2-3 weeks. We want to release the next CR minor shortly after the Kubernetes minor which is scheduled for 27th August

@sbueringer that should be doable. I will work in your remarks today.
Only risk I see, is if I am unable to get the newly added integration test to pass. I added a separate testenv to test the new and old APIs sending events in isolation from each other.
This worked locally on my machine but seems to timeout in CI.

Maybe you have a suggestion there:

controller-runtime/pkg/internal/recorder/recorder_integration_test.go

Lines 118 to 127 in 95e07d9

    
           // this test needs its own env for now to not interfere with the previous one. 
        
           // Once the deprecated API is removed this can be removed. 
        
           testenv := &envtest.Environment{} 
        
           cfg, err := testenv.Start() 
        
           Expect(err).NotTo(HaveOccurred()) 
        
           defer testenv.Stop() //nolint:errcheck 
        
           clientset, err := kubernetes.NewForConfig(cfg) 
        
           Expect(err).NotTo(HaveOccurred())

sbueringer · 2025-08-12T13:52:18Z

Maybe you have a suggestion there:

Very quick first look. The error seems to be in another test:

"Manager Suite: [It] manger.Manager should not leak goroutines if the default event broadcaster is used & events are emitted" (pkg/manager/manager_test.go:1900)

sbueringer · 2025-08-12T13:54:33Z

I'm wondering if the client we are using in this test case is using the clientTransport for which we are force closing idle connections here:

controller-runtime/pkg/manager/manager_test.go

Line 1947 in bc36d51

clientTransport.CloseIdleConnections()

(I think it should, but maybe it's somehow not)

sbueringer · 2025-08-12T13:57:28Z

/retest

I think one of the two failed test cases is an unrelated flake

Signed-off-by: Borja Clemente <[email protected]>

The StartEventWatcher call is no longer needed since calling StartRecordingToSink already sets up a watcher and manages it via its context. Signed-off-by: Borja Clemente <[email protected]>

Signed-off-by: Borja Clemente <[email protected]>

- Remove outdated comment - Improve error handling when setting default Options - Replace deprecated sink recording function - log errors from StartEventWatcher Signed-off-by: Borja Clemente <[email protected]>

clebs · 2025-08-12T21:29:55Z

/retest

Signed-off-by: Borja Clemente <[email protected]>

clebs · 2025-08-12T22:06:51Z

/retest

k8s-ci-robot · 2025-08-12T22:26:33Z

@clebs: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-controller-runtime-test	`f06675e`	link	true	`/test pull-controller-runtime-test`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

clebs · 2025-08-12T22:34:26Z

I have worked in all the feedback, thanks!

I will try to figure out what is wrong with the tests. This new test is definitely affecting other suites somehow.
I did try many different approaches to test the new API without having a separate testenv, but none worked.

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 22, 2025

k8s-ci-robot requested review from JoelSpeed and sbueringer July 22, 2025 20:32

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jul 22, 2025

k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jul 22, 2025

clebs changed the title ~~⚠ Migration to the new events API~~ ⚠️ Migration to the new events API Jul 22, 2025

clebs mentioned this pull request Jul 22, 2025

Switch to use events.EventRecorder instead of record.EventRecorder #2141

Open

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jul 22, 2025

clebs force-pushed the new-events-api branch from 732ed05 to c8698d8 Compare July 23, 2025 15:49

alvaroaleman reviewed Jul 23, 2025

View reviewed changes

clebs force-pushed the new-events-api branch 2 times, most recently from 6bfbd83 to 0528c33 Compare July 24, 2025 12:54

valdar reviewed Jul 25, 2025

View reviewed changes

pkg/internal/recorder/recorder.go Outdated Show resolved Hide resolved

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 29, 2025

clebs force-pushed the new-events-api branch from 592ade2 to 4541c71 Compare July 30, 2025 14:02

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 30, 2025

clebs marked this pull request as ready for review August 7, 2025 09:26

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 7, 2025

k8s-ci-robot requested review from alvaroaleman and troy0820 August 7, 2025 09:26

clebs force-pushed the new-events-api branch from b1e2d6c to bc36d51 Compare August 7, 2025 09:31

k8s-ci-robot assigned sbueringer Aug 7, 2025

sbueringer reviewed Aug 12, 2025

View reviewed changes

clebs added 11 commits August 12, 2025 23:17

Base migration to the new events API

a460c4f

Signed-off-by: Borja Clemente <[email protected]>

Add support for both the new and old events APIs simultaneously.

032b80c

Signed-off-by: Borja Clemente <[email protected]>

Adapt StartEventWatcher to the new API

c91e239

Signed-off-by: Borja Clemente <[email protected]>

Revert renaming of GetEventRecorderFor

fa9c081

Signed-off-by: Borja Clemente <[email protected]>

Remove StartEventWatcher call

709d552

The StartEventWatcher call is no longer needed since calling StartRecordingToSink already sets up a watcher and manages it via its context. Signed-off-by: Borja Clemente <[email protected]>

Fix linting issues

047d2b2

Signed-off-by: Borja Clemente <[email protected]>

Improve message when no action is provided for an event.

be84c84

Signed-off-by: Borja Clemente <[email protected]>

Support both events APIs

03d8e1a

Signed-off-by: Borja Clemente <[email protected]>

Add unit tests for the new events API

adfebed

Signed-off-by: Borja Clemente <[email protected]>

Fix new API not broadcasting events and add integration test

c0481cb

Signed-off-by: Borja Clemente <[email protected]>

Apply review feedback

70be4c0

- Remove outdated comment - Improve error handling when setting default Options - Replace deprecated sink recording function - log errors from StartEventWatcher Signed-off-by: Borja Clemente <[email protected]>

clebs force-pushed the new-events-api branch from 95e07d9 to 70be4c0 Compare August 12, 2025 21:17

Log errors from StartRecordingToSinkWithContext

f06675e

Signed-off-by: Borja Clemente <[email protected]>

⚠️ Migration to the new events API #3262

Are you sure you want to change the base?

⚠️ Migration to the new events API #3262

Conversation

clebs commented Jul 22, 2025

Uh oh!

k8s-ci-robot commented Jul 22, 2025

Uh oh!

clebs commented Jul 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alvaroaleman commented Jul 22, 2025

Uh oh!

alvaroaleman left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alvaroaleman Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

clebs Jul 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

clebs Aug 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sbueringer commented Aug 7, 2025

Uh oh!

k8s-ci-robot commented Aug 11, 2025

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

sbueringer commented Aug 12, 2025

Uh oh!

clebs commented Aug 12, 2025

Uh oh!

sbueringer commented Aug 12, 2025

Uh oh!

sbueringer commented Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sbueringer commented Aug 12, 2025

Uh oh!

clebs commented Aug 12, 2025

Uh oh!

clebs commented Aug 12, 2025

Uh oh!

k8s-ci-robot commented Aug 12, 2025

Uh oh!

clebs commented Aug 12, 2025

Uh oh!

Uh oh!

clebs commented Jul 22, 2025 •

edited

Loading

alvaroaleman Jul 23, 2025 •

edited

Loading

clebs Jul 24, 2025 •

edited

Loading

clebs Aug 7, 2025 •

edited

Loading

sbueringer commented Aug 12, 2025 •

edited

Loading