arXiv:2608.22543 2026 Dynamics 2 ideas extracted · analyzed Aug 29, 2026
The paper provides quantitative control of random matrix products through Lyapunov exponents and their gaps. This is transferable to recurrent networks and state-space models because long-horizon gradient behavior is determined by products of transition Jacobians, not only by the spectrum of individual layers. A practical adaptation is to estimate finite-time Lyapunov exponents with QR iterations and regularize adjacent exponent gaps while constraining the largest exponent. The paper also gives a concentrated stationary distribution for normalized two-block energies, which can motivate state initialization and normalization based on the recurrent dynamics' projective fixed point.
Regularize a recurrent or state-space model using finite-time Lyapunov exponents of its actual hidden-state transition products. Penalize collapsed adjacent exponents while also controlling the largest exponent, encouraging several useful state directions instead of one dominant direction or universal contraction.
Split a recurrent state into two blocks and initialize their variances and cross-correlation according to the stationary projective energy distribution induced by the transition. This places the initial hidden state near the typical invariant direction of the dynamics instead of forcing a long transient from zero or isotropic noise.