arXiv:2608.10241 2026 Regularization 1 ideas extracted · analyzed Sep 1, 2026
The paper introduces a direction-dependent subgaussian geometry for a convex body: each projection is scored by the ratio between its ψ2-norm and its L2-norm. This suggests a neural-network regularizer that controls heavy-tailed feature directions rather than merely penalizing variance or individual large activations. The direct transfer is to whiten a hidden representation, learn an orthogonal channel rotation, and minimize the worst empirical subgaussian ratio across channels; this preserves second-order scale while improving tail behavior. The extracted material does not expose the paper's sharper volume-ratio or basis-construction constants, so the proposed experiment relies only on the explicit ψ2 machinery provided.
Add a learnable orthogonal rotation to a hidden representation and train it to make every channel projection have a small ψ2/L2 ratio. Unlike variance normalization, this explicitly suppresses directions with unusually heavy empirical tails while preserving the total quadratic energy of the representation.