Spectral phase transitions in Gaussian multi-index models

What the math gives to ML

The paper gives a principled spectral mechanism for recovering nonlinear task-relevant subspaces from Gaussian covariates: responses are used to reweight rank-one covariate matrices, and informative eigenvalues separate from a random-matrix bulk only above a sharp sample-to-dimension threshold. This can become a supervised feature-learning initializer for neural networks, especially when the target depends on a small set of nonlinear projections and ordinary gradient descent starts in an uninformative high-dimensional basin. The most direct implementation is to construct a bounded label-dependent lifted covariance, extract its outlier eigenspace, reshape it into an estimate of the latent input subspace, and initialize or constrain the first layer with that estimate. The paper's matrix-valued preprocessing also suggests using multiple response statistics jointly rather than independently fitting scalar feature scores.