Fourth-Moment Geometry of Rademacher Sums

What the math gives to ML

The paper identifies the fourth-order mass \(\sum_i a_i^4\) as the quantity controlling how a normalized Rademacher sum departs from Gaussian behavior, rather than treating all coefficient vectors with the same \(\ell_2\) norm as equivalent. This gives neural-network designers a directly computable effective-dimension signal for randomized signed projections, perturbations, and quantization noise. The most practical transfer is to regularize or constrain coefficient vectors in signed linear operators so that their fourth-order mass matches a target: low mass produces more Gaussian-like, dimension-spread noise, while high mass deliberately preserves sparse, heavy-tailed behavior. A small-scale ablation can test whether this improves stability and accuracy of sign-randomized layers at equal compute.