Sharp Tail Bounds Beyond Twice the Mean
arXiv:2608.06317
2026
Regularization
1 ideas extracted · analyzed Aug 31, 2026
What the math gives to ML
The paper gives a sharp worst-case upper-tail bound for sums of independent nonnegative variables with bounded means, identifying sparse Bernoulli spikes at the threshold as the extremal distribution. The transferable asset is a distributionally robust certificate: rare, full-sized spikes are harder to control than smooth fluctuations when only nonnegativity, independence, and means are known. A neural network can use the bound as a differentiable regularizer for independently sampled branches, activation loads, or compute allocations. The method is most appropriate when component means are analytically controlled or conservatively upper-bounded.
Ideas from this paper
Unverified
2026
Apply the paper's extremal tail bound to independently sampled nonnegative neural-network contributions, such as stochastic-depth branch activations, independently gated expert loads, or separately allocated memory chunks. Penalize the analytic worst-case probability that their sum exceeds a budget, using the fact that the worst admissible distribution is a sparse Bernoulli spike at the threshold.
Useful5/10
Difficulty4/10
Novelty8/10