Stationary covariance spectra of discrete-time non-normal random recurrent dynamics

What the math gives to ML

The paper isolates a failure mode of random recurrent dynamics that is invisible from eigenvalues alone: a stable, strongly non-normal recurrent matrix can produce a highly anisotropic stationary covariance and concentrate activity into a few principal components. The transferable object is the stationary covariance equation, whose spectrum directly measures how recurrent amplification distributes noise and inputs across hidden directions. A practical neural-network adaptation is to monitor or estimate this covariance spectrum in an RNN and penalize excessive spectral concentration, rather than controlling only the spectral radius of the recurrent matrix. This should improve stability and usable hidden-state dimensionality without requiring the recurrent matrix itself to be normal or orthogonal.