Skip to content

Conversation

nirrozenbaum
Copy link
Contributor

@nirrozenbaum nirrozenbaum commented Sep 10, 2025

What type of PR is this?
/kind flake

What this PR does / why we need it:
Fix a test flake of random weighted picker.
This fix is based on expected probabilities with some tolerance to make sure the actual random sampling is close enough to the expected results. more on the numbers:

🔹 Setup

  • Suppose expected probability = 𝑝.
  • Number of trials = 𝑛 = 10000
  • The observed count follows a binomial distribution:
    𝑋∼Binomial(𝑛,𝑝)
  • Observed frequency = 𝑋/𝑛
  • Standard deviation (σ) of observed frequency is:
    𝜎=sqrt(𝑝(1−𝑝)/𝑛)

🔹 Example: worst-case variance

The variance is largest at 𝑝=0.5

𝜎=sqrt(0.5⋅0.5/10000)≈0.005

So the natural “jitter” is about ±0.5 percentage points.

🔹 With tolerance = ±5% (0.05)

  • Allowed range = expected ±0.05.
  • That is 10× wider than σ.

In normal distribution terms, that’s a 10σ bound.
The probability of going beyond 10σ is about 10^-23 — practically impossible in the lifetime of the universe.

Which issue(s) this PR fixes:

Fixes #1538

Does this PR introduce a user-facing change?:

None

@k8s-ci-robot k8s-ci-robot added the kind/flake Categorizes issue or PR as related to a flaky test. label Sep 10, 2025
Copy link

netlify bot commented Sep 10, 2025

Deploy Preview for gateway-api-inference-extension ready!

Name Link
🔨 Latest commit 5c60ea0
🔍 Latest deploy log https://app.netlify.com/projects/gateway-api-inference-extension/deploys/68c190b5a15b7d0008b7dca0
😎 Deploy Preview https://deploy-preview-1561--gateway-api-inference-extension.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Sep 10, 2025
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: nirrozenbaum

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Sep 10, 2025
@nirrozenbaum
Copy link
Contributor Author

/cc @kfswain

@elevran
Copy link
Contributor

elevran commented Sep 10, 2025

I can't comment on the math (i.e., is the distribution of samples normal or binomial, should we be looking for actual P vs expected P to be within 5% based on the expected P or as absolute,...) but regarding the code:
/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 10, 2025
@k8s-ci-robot k8s-ci-robot merged commit 7a979da into kubernetes-sigs:main Sep 10, 2025
11 checks passed
@nirrozenbaum nirrozenbaum deleted the picker-flake branch September 10, 2025 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/flake Categorizes issue or PR as related to a flaky test. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test flake in weighted random picker

3 participants