ML: Normalization

Machine-learning ideas tagged Normalization in the ML taxonomy of the Math2NN corpus.

2 ideas found

ML: Normalization Math: Statistics Clear all

Unverified 2026

Gain-aligned branch shunting

Replace an additive nonnegative feature readout by several local divisive branches, where each branch divides a signal pathway by a positive pool chosen to estimate shared multiplicative gain. Initialize or constrain each pool toward the dominant nuisance covariance direction while retaining an additive bypass so the model can reject harmful normalization. This should improve robustness when nuisance gain is shared across features, but not when the pool support is shuffled or its measurements…

Useful6/10

Difficulty5/10

Novelty5/10

Architecture Accuracy Sample efficiency Stability

Paper: When Branch-Local Shunting Helps: A Gain-Load-Alignment Principle for Dendritic E/I Networks arXiv:2607.24990

Unverified 2026

Sharp thin-shell representation regularizer

Add a radial-fluctuation penalty to a feature layer after explicitly centering and whitening its activations across the minibatch. The paper supplies an interpretable threshold, eight times the feature dimension, for the variance of squared feature norms. The penalty activates only when empirical radial variance exceeds that threshold, avoiding unnecessary pressure toward constant-norm representations.

Useful5/10

Difficulty5/10

Novelty6/10

Regularization Stability Generalization Embedding

Paper: Digesting the proof of the sharp thin-shell inequality arXiv:2607.23307