Moment comparisons, Sudakov inequalities and entropy of centroid bodies

arXiv:2608.10853 2026 Regularization 1 ideas extracted · analyzed Sep 1, 2026

What the math gives to ML

The paper provides explicit comparisons between moments of any gauge under an isotropic log-concave distribution and under a Gaussian reference, with a dimension-dependent but nonasymptotic moment window. The transferable asset is a principled way to constrain feature-norm tails after whitening: low and moderate moments should remain within a controlled factor of Gaussian moments, while excessively large orders are avoided. This suggests a cheap activation regularizer that monitors a small set of feature moments and penalizes departures from the theorem's admissible window, with the theorem serving as a calibration target rather than an unconditional guarantee because neural activations need not be log-concave.

Ideas from this paper

Unverified 2026

Gaussian Moment-Window Activation Regularizer

Whiten intermediate feature vectors and constrain several gauge moments to remain in the dimension-dependent interval predicted by the paper's Gaussian/log-concave comparison. Apply the penalty only to moderate orders, where the paper gives a uniform bound independent of the particular log-concave distribution; this should suppress heavy activation tails without forcing all features to be exactly Gaussian.

Useful5/10
Difficulty3/10
Novelty6/10
Paper: Moment comparisons, Sudakov inequalities and entropy of centroid bodies arXiv:2608.10853