Spectral Characterisation of Covariance Existence in Markov-Switching Affine Recurrences
arXiv:2607.09994
2026
Dynamics
2 ideas extracted · analyzed Aug 30, 2026
What the math gives to ML
The paper isolates a finite-dimensional mean-square stability condition for affine systems whose linear dynamics switch according to a finite-state Markov chain. The key transferable asset is that almost-sure stability of random matrix products is insufficient to prevent exploding second moments; the relevant object is an explicit block Kronecker operator whose spectral radius controls variance. This gives neural architectures with discrete routing or mode switching a computable stability certificate, rather than relying only on gradient-based empirical stabilization. The most direct transfer is to recurrent, state-space, or mixture-of-experts modules: constrain the mode-transition Kronecker operator during training and optionally solve its linear moment equations to initialize or normalize hidden-state covariances.
Ideas from this paper
✓✓ Beats tuned baseline
2026
Replace an unconstrained recurrent or state-space transition with a finite set of mode matrices selected by a Markov routing process, while explicitly constraining the associated Kronecker operator to have spectral radius below one. This targets exploding hidden-state variances caused by rare but repeatedly visited unstable modes, a failure mode not detected by average spectral radius or ordinary Lyapunov stability.
Useful8/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Use the paper's finite-dimensional second-moment equations to compute the stationary covariance induced by a Markov-switched recurrent layer before training, then whiten or scale each mode's hidden state using that covariance. This can prevent mode-specific saturation and eliminate a long burn-in period in long-context RNNs and state-space models.
Useful7/10
Difficulty6/10
Novelty7/10