Moment comparisons, Sudakov inequalities and entropy of centroid bodies
arXiv:2608.10853
2026
Regularization
1 ideas extracted · analyzed Sep 1, 2026
What the math gives to ML
The paper provides explicit comparisons between moments of any gauge under an isotropic log-concave distribution and under a Gaussian reference, with a dimension-dependent but nonasymptotic moment window. The transferable asset is a principled way to constrain feature-norm tails after whitening: low and moderate moments should remain within a controlled factor of Gaussian moments, while excessively large orders are avoided. This suggests a cheap activation regularizer that monitors a small set of feature moments and penalizes departures from the theorem's admissible window, with the theorem serving as a calibration target rather than an unconditional guarantee because neural activations need not be log-concave.
Ideas from this paper
Unverified
2026
Whiten intermediate feature vectors and constrain several gauge moments to remain in the dimension-dependent interval predicted by the paper's Gaussian/log-concave comparison. Apply the penalty only to moderate orders, where the paper gives a uniform bound independent of the particular log-concave distribution; this should suppress heavy activation tails without forcing all features to be exactly Gaussian.
Useful5/10
Difficulty3/10
Novelty6/10