Extreme least singular values of Gaussian row submatrices and a phase retrieval stability problem
arXiv:2607.06249
2026
Regularization
2 ideas extracted · analyzed Aug 30, 2026
What the math gives to ML
The paper quantifies a severe worst-case conditioning phenomenon: among all square row subsets of an overcomplete Gaussian matrix, the smallest least singular value is exponentially small, with exponent determined by the combinatorial entropy h(\gamma). This is directly relevant to neural measurement layers, feature projections, token/channel erasure, and models expected to remain invertible after structured row dropout. The transferable asset is the extreme-subset viewpoint: average singular-value conditioning can look excellent while one of exponentially many subsets is nearly singular. A practical adaptation is to train or select linear layers using sampled worst-case subset conditioning, using the theorem as a baseline for how much redundancy is needed.
Ideas from this paper
△ Mechanism confirmed, baseline not beaten
2026
Replace an unconstrained overcomplete linear measurement or embedding matrix by one trained to remain well-conditioned after deletion of a prescribed number of rows. The objective explicitly targets the smallest singular value over sampled row subsets, preventing a layer from relying on fragile combinations of features that disappear under channel, sensor, token, or measurement erasures.
Useful7/10
Difficulty5/10
Novelty7/10
Unverified
2026
Turn row dropout into an adversarial conditioning problem rather than independent Bernoulli noise. At each training step, search for a subset of surviving channels or measurements with unusually small least singular value, train the downstream network on that subset, and gradually increase the search strength so training directly exposes failure modes hidden by average-case dropout.
Useful6/10
Difficulty6/10
Novelty8/10