Gaussian Convexity Principles for Sharp Moderate Deviations of Gaussian Maxima and Critical SK Free Energy Variance
arXiv:2607.21392
2026
Stability
1 ideas extracted · analyzed Aug 30, 2026
What the math gives to ML
The paper gives a correlation-sensitive moderate-deviation law for the maximum of a centered Gaussian vector: when the expected maximum is already a nontrivial fraction of the independent-Gaussian scale, upward excursions have a sharper exponent than the generic Borell bound. This can transfer to attention logits and MoE router logits, where correlated scores may occasionally produce one-token or one-expert domination despite marginal normalization. The practical adaptation is an extreme-logit calibration rule that estimates the expected maximum and penalizes or rescales logits above a mathematically motivated threshold. The guarantee is exact only for approximately Gaussian logits with bounded marginal variance, so the proposed experiment should test both Gaussian synthetic logits and trained neural representations.
Ideas from this paper
Unverified
2026
Normalize attention or router logits and control their upper tail using the paper's sharper Gaussian-maximum exponent rather than a correlation-blind sub-Gaussian bound. Use the resulting threshold to add a soft penalty or adaptive temperature whenever the observed maximum exceeds the calibrated level, reducing rare one-token or one-expert domination.
Useful5/10
Difficulty4/10
Novelty6/10