Try to reduce prometheus port conflict test flake#7086
Try to reduce prometheus port conflict test flake#7086jack-berg wants to merge 2 commits intoopen-telemetry:mainfrom
Conversation
…y-java into fix-prometheus-port-conflict
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #7086 +/- ##
============================================
- Coverage 89.86% 89.84% -0.02%
+ Complexity 6613 6612 -1
============================================
Files 740 740
Lines 19991 19991
Branches 1966 1966
============================================
- Hits 17964 17961 -3
- Misses 1437 1439 +2
- Partials 590 591 +1 ☔ View full report in Codecov by Sentry. |
|
heads up, I hit rerun, but it looks like it flaked beforehand: https://github.com/open-telemetry/opentelemetry-java/actions/runs/13207505490/job/36923855747?pr=7086 |
| server -> { | ||
| assertThat(server.getAddress().getHostName()).isEqualTo("0:0:0:0:0:0:0:0"); | ||
| assertThat(server.getAddress().getPort()).isEqualTo(9464); | ||
| assertThat(server.getAddress().getPort()).isPositive(); |
There was a problem hiding this comment.
If this fails, I'm going to have questions.. ;)
@trask what strategies does the instrumentation repo use for flaky tests? Ideally, I'd like to just be able to retry select tests a second time. The probability of failing twice in a row should be low enough to be tolerable. |
|
|
Closing in favor of #7106 |
Example flake here: https://github.com/open-telemetry/opentelemetry-java/actions/runs/13184973874/job/36804837095?pr=7055#step:6:1659
I believe I've only noticed these on windows. Not sure why. Maybe the way windows assigns random available port is more prone to race conditions / more deterministic than other OS. I.e. perhaps between the time we ask for a available port and initialize a prometheus server with that port, another test initializes a different prometheus server with that same port.
Trying to mitigate that by adjusting tests to favor setting the port to
0, causing the implementation to assign an available port. And then adjusting test code flow to make assertions of this port.