Extreme least singular values of random row submatrices with bounded-density subgaussian entries

What the math gives to ML

The paper gives a sharp exponential law for the worst-conditioned square row submatrix of a tall random matrix: among exponentially many subsets, the minimum singular value is typically of order $e^{-h(\gamma)m}$, where $h(\gamma)$ is the combinatorial entropy of choosing $m$ rows from $N\approx\gamma m$. This transfers to robustness against neuron removal, feature dropout, or routing failures in overcomplete layers. A practical adaptation is to regularize sampled row subsets toward larger smallest singular values, while using the theorem's entropy exponent as a random-matrix baseline for evaluating whether learned redundancy is genuinely robust.