Aggregation with Exponential Weights is Optimal in Expectation

What the math gives to ML

The paper gives a sharp, distribution-free guarantee for exponential-weight aggregation under bounded, Lipschitz, strongly convex losses: a fixed temperature is sufficient once it crosses an explicit threshold, and the excess risk is then of order T log(M)/(n+1). The transferable asset is not softmax weighting itself, which is standard, but the quantitative temperature condition and the demonstrated phase transition between too-cold and sufficiently-hot aggregation. A practical neural-network use is to aggregate a finite set of checkpoints, adapters, subnetworks, or experts with a temperature selected from loss-range and curvature estimates rather than tuned blindly. This is most promising as a calibration rule for ensemble or MoE-style gates where overconfident low-temperature weights cause premature expert collapse.