Skip to content

Commit 88a4fe8

Browse files
authored
Make seedless test deterministic (#64)
* Update test without seed to avoid coin flip * Fix comment * Another comment tweak
1 parent 68471a1 commit 88a4fe8

File tree

1 file changed

+6
-2
lines changed

1 file changed

+6
-2
lines changed

tests/basemath_analysis/test_basemath.py

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -166,8 +166,12 @@ def test_seed_not_provided():
166166
"""
167167
basemath = BaseMathsTest(0.3, 0.9, 0.05, 0.2)
168168
assert basemath.required_samples == 42
169-
assert basemath.evaluate_experiment(0, -1, 0, 10) == 0
170-
assert basemath.evaluate_experiment(-1, -10, 10, 40) == -1
169+
# First evaluation to show that the experiment starts inconclusive. Using a number of samples that's greater than
170+
# zero but less than the total requirement risks losing the 'coin flip' to determine whether we crossed the line
171+
# (since we don't control the seed in this test), so we set it to zero to keep the test deterministic
172+
assert basemath.evaluate_experiment(0, 0, 0, 0) == 0
173+
# And now we evaluate with >=required_samples, again deterministic
174+
assert basemath.evaluate_experiment(0, -10, 0, 50) == -1
171175

172176

173177
def test_all_zeros_in_experiment_evaluation():

0 commit comments

Comments
 (0)