Samuels Chance-Budget Regularizer

Integrate this into stochastic compute control, preferably a small MoE or dynamic-depth Transformer. Define X_i for one example as the nonnegative number of active tokens routed to expert i, activation bytes used by layer i, or FLOPs incurred by optional block i. For a simple MVP, use stochastic depth: X_i = c_i B_i, where c_i is the measured cost of block i and B_i is its Bernoulli execution indicator. On every minibatch, measure x_i for each component and update a running mean with mu_i <- beta mu_i + (1-beta) mean_batch(x_i). Sort the means into ascending order and retain the sorting permutation. Compute D_i = lambda - cumulative_sum(mu_sorted before i), then q_i = product over j from i to n of (1 - mu_sorted[j] / D_i). If lambda <= sum_i mu_i, increase lambda or add a separate overload penalty; never apply the theorem outside its stated condition. Add alpha times L_budget to the task loss, using the softmin formula during training. Pseudocode: collect costs x; mu=EMA(mu,mean(x)); m,perm=sort(mu); D[i]=lambda-sum(m[:i]); q[i]=prod(1-m[i:]/D[i]); cert=softmin(q,tau); loss=task_loss-alpha*log(cert); backpropagate and update parameters. The theorem supplies the exact mean-only certificate; independence and the quality of the EMA estimates must be checked empirically. First experiment: train a 4-layer stochastic-depth MLP or tiny Transformer on CIFAR-10 or WikiText-2 at a fixed expected compute budget. Compare no regularizer, a mean-cost penalty, and this regularizer. Record empirical P(total_cost < lambda), worst-batch overload probability, accuracy or perplexity, and training variance. Success means higher budget satisfaction at matched accuracy and a useful certificate that remains conservative under distribution shift.