Characterizing Bias in Post-Bandit Inference under Index Algorithms

What the math gives to ML

This paper gives a sharp, algorithm-dependent characterization of post-adaptive-sampling bias rather than treating bandit samples as iid. The transferable object is the effective exploration rate: for generalized UCB indices x + f_t/sqrt(n), the standardized sample-mean bias is approximately -1/f_T for every arm that is not uniquely optimal, so the exploration schedule directly determines downstream target bias. A practical neural-network use is to correct targets or losses in contextual-bandit and active-data-collection pipelines, where a shared encoder or reward model is trained on arm-conditioned samples produced by UCB-like policies. The correction should be uncertainty-aware because the theorem has a different leading term for a uniquely optimal arm.