You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Due to the small size of the dataset used for test purposes, training
on MedCAT fails intermittently. This is due to the fact that after
splitting the dataset into training and testing sets, the training set
might end up being empty. Observation shows that this happens ~40% of
the time for our test dataset and the default test size of 0.2. With
that in mind, we rerun the flaky test up to 6 times before failing,
based on the following calculation:
P(failure) = ~0.4
P(n failures) = P(failure) x P(failure) x ... x P(failure) = 0.4^n
If we want to keep the probability of failure below 0.01:
0.4^n < 0.01 => log(0.4^n) < log(0.01) => n x log(0.4) < log(0.01) =>
n > log(0.01) / log(0.4) => n > 5.03
Signed-off-by: Phoevos Kalemkeris <[email protected]>
0 commit comments