Slack-aware anytime stopping
Implementation & benchmark of arXiv:2607.04620 — The Exact Ville Identity: From the Absorbing Case to the General Law with an Application to E-Values
Source paper: The Exact Ville Identity: From the Absorbing Case to the General Law with an Application to E-Values arXiv:2607.04620 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Attach a nonnegative e-process to a held-out stream used to monitor adaptively chosen neural-network checkpoints. Instead of using only Ville's conservative threshold b = 1/α, estimate overshoot, drift loss, and surviving mass, then test whether a conservative version of the exact identity permits earlier detection at the same empirical type-I error.
Formulas
Mathematical statement
Let (M_n, F_n) be a nonnegative supermartingale with M_0 = 1, meaning M_n ≥ 0 and E[M_n | F_(n-1)] ≤ M_(n-1). For b > 1, define T_b = inf{n ≥ 1 : M_n ≥ b}, with T_b = infinity when there is no crossing. The exact identity is P(T_b < infinity) = (1 − D_b − R_b) / (b + O_b). The overshoot is O_b = E[(M_(T_b) − b) 1{T_b < infinity}]. The cumulative supermartingale loss is D_b = E[sum from n=1 to T_b of (M_(n-1) − E[M_n | F_(n-1)]) 1{T_b ≥ n}]. The survival residual is R_b = E[M_infinity 1{T_b = infinity}], where M_infinity is the limiting process value on noncrossing paths. A concrete neural evaluator can use M_n = product from i=1 to n of (1 + η_i X_i), where X_i is a bounded improvement score, 0 ≤ η_i ≤ 1 is predictable, and E[X_i | F_(i-1)] ≤ 0 under the null. For target crossing probability p, the identity suggests b_recal = (1 − D_b − R_b)/p − O_b. In deployment, replace unknown terms by conservative confidence bounds rather than optimistic point estimates.
Implementation notes
(1) Integration point: add a sequential evaluator next to training, fine-tuning, or RLHF. At each adaptively selected checkpoint, evaluate the current model on a fresh validation batch. Define X_i in [−1,1] as a clipped improvement score relative to a fixed null model, baseline policy, or pretraining checkpoint. The validation stream must not be used to choose score calibration or checkpoint scheduling.
(2) Pseudocode:
M = 1
for each fresh batch i:
x = clip(improvement_score(model_i, batch_i), -1, 1)
eta = predictable_step_size(i, previous_batches)
M_prev = M
M = M_prev * (1 + eta * x)
if M >= b:
record overshoot M - b
reject or stop
break
if no crossing:
record terminal mass M
(3) Estimate O_b from crossing streams, R_b from terminal M values on noncrossing streams, and D_b from a cross-fitted estimate of the conditional null mean. If a fitted model estimates μ_i = E[X_i | F_(i-1)], estimate the one-step loss contribution using max(0, −M_(i-1) η_i μ_i). Use bootstrap or concentration upper bounds for D_b and R_b, and an upper confidence bound for O_b when computing a conservative threshold. Do not use optimistic plug-in estimates as a formal validity guarantee.
(4) First experiment: use a small ResNet on CIFAR-10 or a 1–10M parameter language model, create 200 independent held-out streams, and compare standard Ville stopping at b = 1/α with pilot-estimated recalibration. Under a null, freeze the model or compare identical checkpoints; under an alternative, train a model with known improvement. Measure empirical false-crossing probability, validation examples consumed before detection, and detection power. Success means at least 20% fewer validation examples or earlier detection at matched false-alarm rate, while the conservative rule remains below the target α across repeated seeds.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.