betting_mart should not ignore stalls#107
Conversation
|
It looks like, from the comments, that line was supposed to set the terms in the P-value sequence equal to 1, which (I think) is correct logic. But instead it sets the terms of the martingale itself equal to 1, which I agree it shouldn't do. I'm wondering if anything is lost by removing the line entirely. In assuming a small rate of 2-vote-overstatements, Nonneg_mean.optimal_comparison() defaults to a bet slightly below the maximal bet, so the martingale should never go completely broke even if a 0 (e.g., a 2-vote-overstatement) is observed. |
|
i agreee if 2-vote overstatement rate > 0, lamda < 1/mj in optimal_comparison() and you should never see the problem. I was exploring "what if we assume there are no errors and adapt as we see them?" betting strategy and inadvertantly triggered it. Im not sure why the pvalue goes to 1 when the martingale vanishes? |
|
I see that youve already made all your martingale values in so modifying one of the terms doesnt affect subsequent ones. Now I see how that changes everything. Im doing "one sample at a time" and terminating when p < risk. Still not sure why pvalue = 1 when mart = 0. |
|
If you think of the martingale as the wealth of a gambler in a series of
fair or sub – fair bets, the audit of that assertion stops only when the
gambler’s fortune gets to be one over the risk limit (or more). The gambler
cannot borrow money. The gambler can only bet a fraction of their current
wealth on the next draw. If the gambler ever goes broke, their fortune can
never grow again: the P-value is one. That’s why the methods for choosing
the bet methods stop short of betting 100% of the gambler‘s current
fortune. In a comparison audit, the only time you should bet 100% of your
stake is if you are absolutely certain every CVR is correct. If there’s
even one overstatement error in the sample, it’s game over for the auditor:
they go broke and have to do a full hand count.
Philip B. Stark | Distinguished Professor, Department of Statistics |
University of California
Berkeley, CA 94720-3860 | 510-394-5077 | www.stat.berkeley.edu/~stark |
@philipbstark
…On Fri, Jan 2, 2026 at 12:43 John Caron ***@***.***> wrote:
*JohnLCaron* left a comment (pbstark/SHANGRLA#107)
<#107 (comment)>
I see that youve already made all your martingale values in
np.cumprod(1 + lam * (x - m))
so modifying one of the terms doesnt affect subsequent ones.
Now I see how that changes everything. Im doing "one sample at a time" and
terminating when p < risk.
Sorry, Im not used to python array processing.
Still not sure why pvalue = 1 when mart = 0.
—
Reply to this email directly, view it on GitHub
<#107 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AANFDWPD4GAD67P37B6ANET4E3J7JAVCNFSM6AAAAACPSTMT62VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTOMBWGE4TKOJUG4>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
|
Ok, I see. Setting the terms to 1 when the martingale vanishes is a way to set pvalue to 1. However, perhaps its misleading when you are returning the martingale sequence, so you might consider only doing so when out = "p-values". I have not been setting 1.0 as the maximum value of pvalue. Ive really been using it to mean 1/term, which may be greater than 1. The only place that comes into play I think is for multi-round audits. But subsequent rounds just start from where the previous ones stop, and 1/term gives you the sense of "how much further there is to go". In practice, the rounds are independent of each other, that is, they start from pvalue = 1 and replicate the previous round exactly up to the previous sample size, and then keep going with the new samples. Ive been assuming that I cant use the measured error rates from the previous round when auditing, because it would be a form of "peeking ahead". However I can use them when estimating the next round's sample size. If any of those assumptions seem wrong, your thoughts would be appreciated. -- John |
3 participants
This line of code in betting_mart intervenes when a term gets close to 0, by changing the term to 1, effectively ignoring the stall.
This happens when x = 0 and lamda is ~= 1/mj ~= 2. The effect is to ignore x = 0, which is the worst assort value; ignoring it makes the audit invalid I think.
A better way to prevent stalls is to limit the value of lamda to < 1/mj.
Im planning on studying how much less it should be, since keeping lamda close to 2 will optimize the no error case.
Its possible that your bet function already enforces lamda < 1/mj, in which case you would never see this problem.
However, I was naively using a port of Nonneg_mean.optimal_comparison() for the OneAudit bet function, and was getting suspiciously good results. Once I found that problem, the results were less optimal.
In any case I cant see any good reason that the line should be in there.