Gaussian Convexity Principles for Sharp Moderate Deviations of Gaussian Maxima and Critical SK Free Energy Variance

arXiv:2607.21392 2026 Stability 1 ideas extracted · analyzed Aug 30, 2026

What the math gives to ML

The paper gives a correlation-sensitive moderate-deviation law for the maximum of a centered Gaussian vector: when the expected maximum is already a nontrivial fraction of the independent-Gaussian scale, upward excursions have a sharper exponent than the generic Borell bound. This can transfer to attention logits and MoE router logits, where correlated scores may occasionally produce one-token or one-expert domination despite marginal normalization. The practical adaptation is an extreme-logit calibration rule that estimates the expected maximum and penalizes or rescales logits above a mathematically motivated threshold. The guarantee is exact only for approximately Gaussian logits with bounded marginal variance, so the proposed experiment should test both Gaussian synthetic logits and trained neural representations.

Ideas from this paper

Unverified 2026

Gaussian Extreme-Logit Calibration

Normalize attention or router logits and control their upper tail using the paper's sharper Gaussian-maximum exponent rather than a correlation-blind sub-Gaussian bound. Use the resulting threshold to add a soft penalty or adaptive temperature whenever the observed maximum exceeds the calibrated level, reducing rare one-token or one-expert domination.

Useful5/10
Difficulty4/10
Novelty6/10
Paper: Gaussian Convexity Principles for Sharp Moderate Deviations of Gaussian Maxima and Critical SK Free Energy Variance arXiv:2607.21392