A Two-regime Khintchine Inequality and an Improved Bound on the Degree-1 Fourier Weight for Linear Threshold Functions

arXiv:2608.27908 2026 Regularization 1 ideas extracted · analyzed Sep 1, 2026

What the math gives to ML

The paper provides a quantitative certificate that a Rademacher linear form cannot remain close to its minimum expected absolute magnitude unless its weight vector is close to a two-coordinate, equal-magnitude extremizer. This can be transferred into a neuron regularizer that discourages degenerate filters or feature projections whose energy is concentrated in only two coordinates, while remaining invariant to signs and permutations. The practical implementation is to estimate expected absolute responses under random sign perturbations and combine them with the paper's explicit distance-sensitive lower bound, yielding a falsifiable feature-diversity and robustness experiment.

Ideas from this paper

Unverified 2026

Khintchine anti-degeneracy regularizer

Add a regularizer that rewards each neuron's expected absolute response to random sign perturbations, normalized by the neuron's l2 norm so ordinary weight scaling cannot trivially increase the objective. Use the paper's distance-sensitive Khintchine lower bound to penalize filters close to the two-coordinate extremal set, promoting distributed and perturbation-stable feature extraction.

Useful5/10
Difficulty3/10
Novelty8/10
Paper: A Two-regime Khintchine Inequality and an Improved Bound on the Degree-1 Fourier Weight for Linear Threshold Functions arXiv:2608.27908