Skip to content

Fix flaky pod Watcher test in target allocator#4180

Merged
swiatekm merged 1 commit intoopen-telemetry:mainfrom
swiatekm:test/fix-flaky-pod-watcher-test
Jul 11, 2025
Merged

Fix flaky pod Watcher test in target allocator#4180
swiatekm merged 1 commit intoopen-telemetry:mainfrom
swiatekm:test/fix-flaky-pod-watcher-test

Conversation

@swiatekm
Copy link
Contributor

@swiatekm swiatekm commented Jul 10, 2025

The root cause of the flakiness was that the fake client we're using doesn't support resource versions. Informers rely on these to avoid race conditions when processing the initial object list. The flaky tests were caused by this race condition. I've fixed this by having the tests check whether the informer event handlers have synced before changing objects using the client.

I've also had the test pass around a pointer to the watcher rather than the whole struct.

Closes #4184.

@swiatekm swiatekm requested a review from a team as a code owner July 10, 2025 10:21
@swiatekm swiatekm added Skip Changelog PRs that do not require a CHANGELOG.md entry flaky test labels Jul 10, 2025
The root cause of the flakiness was that the fake client we're using
doesn't support resource versions. Informers rely on these to avoid race
conditions when processing the initial object list. The flaky tests were
caused by this race condition. I've fixed this by having the tests check
whether the informer event handlers have synced before changing objects
using the client.

I've also had the test pass around a pointer to the watcher rather than
the whole struct.
@swiatekm swiatekm force-pushed the test/fix-flaky-pod-watcher-test branch from 0c89851 to 4eda9c5 Compare July 10, 2025 11:36
@github-actions
Copy link
Contributor

github-actions bot commented Jul 10, 2025

E2E Test Results

 33 files  ±0  221 suites  ±0   3h 51m 55s ⏱️ + 7m 51s
 84 tests ±0   84 ✅ ±0  0 💤 ±0  0 ❌ ±0 
221 runs  ±0  221 ✅ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit 4eda9c5. ± Comparison against base commit 369ee05.

♻️ This comment has been updated with latest results.

@swiatekm swiatekm merged commit c2a7d73 into open-telemetry:main Jul 11, 2025
81 of 84 checks passed
@swiatekm swiatekm deleted the test/fix-flaky-pod-watcher-test branch March 14, 2026 17:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

flaky test Skip Changelog PRs that do not require a CHANGELOG.md entry

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky Unit tests: fail when cluster is running, pass when cluster is deleted

2 participants