On the smallest singular value of the product of random and deterministic matrices
arXiv:2607.06785
2026
Architecture
1 ideas extracted · analyzed Aug 30, 2026
What the math gives to ML
The paper gives a nonasymptotic lower-tail bound for the smallest singular value of a random matrix after deterministic left preconditioning. Its transferable asset is the explicit dependence on the inverse Hilbert–Schmidt norm of the deterministic factor, which quantifies how anisotropic feature mixers increase the risk of near-singular directions. A practical neural-network use is to regularize or initialize deterministic channel-mixing matrices so that randomized linear layers and their local Jacobians retain a reliable minimum gain.
Ideas from this paper
Unverified
2026
Factor a neural linear layer as W = M A, where A is randomized at initialization and M is a deterministic channel mixer or learned feature transform. Regularize M toward low inverse-Hilbert–Schmidt norm under a scale constraint, because the paper's theorem predicts that this raises the high-probability lower bound on s_min(W) and reduces near-singular initialization events.
Useful5/10
Difficulty5/10
Novelty6/10