The concentration game: Bayesian updating, regret, and information

What the math gives to ML

The paper provides an information-theoretic view of exponential weights in which the Gibbs/Bayes distribution is not merely a convenient update, but the Bellman equalizer for a KL-constrained adversarial game. The transferable asset is the exact log-partition and relative-entropy accounting: instead of replacing uncertainty by a generic variance bound, one can measure the per-step cumulant-information gap induced by observed losses. A practical neural-network adaptation is an information-budgeted adaptive softmax router or optimizer portfolio, with temperature chosen from the accumulated information gap and explicit correction when the temperature changes. This is most promising for noisy mixture-of-experts routing, ensembles of adapters, or adaptive learning-rate portfolios rather than for replacing ordinary SGD on individual weights.