# Cρ-stable recurrent transition

- ID: 878
- Canonical URL: https://synthcore.org/idea/878/c-stable-recurrent-transition
- API JSON: https://synthcore.org/api/idea/878.json
- API Markdown: https://synthcore.org/api/idea/878.md
- Verification status: unverified
- Source: [arXiv:2607.13794](https://arxiv.org/abs/2607.13794)
- Category: dynamics
- Solves: stability, generalization, accuracy
- ML areas: rnn, ssm, training-dynamics, optimizer
- Math tags: functional-analysis, linear-algebra, dynamical-systems, control-theory
- Ratings: usefulness 6/10; difficulty 5/10; novelty 7/10

## Idea description

Constrain the transition matrix of an RNN or linear state-space model to the paper's class Cρ instead of controlling only its spectral radius or spectral norm. The resulting transition has an explicit dilation certificate and satisfies ∥T^n∥ ≤ ρ for every time horizon, preventing exploding hidden states while retaining nonnormal dynamics that ordinary spectral normalization may remove.

## Mathematical statement

An operator T on a Hilbert space belongs to Cρ if there is a unitary U on a larger space K and an orthogonal projection P_H onto the model state space H such that T^n = ρ P_H U^n|_H for every integer n ≥ 1. Consequently, ∥T^n∥ ≤ ρ because ∥U^n∥ = 1 and ∥P_H∥ = 1. The paper gives the intrinsic certificate I - 2(1 - 1/ρ) Re(z̅T) - (2/ρ - 1)|z|^2 T*T ≽ 0 for every complex z with |z| ≤ 1, where I is the identity, T* is the conjugate transpose, Re(B) = (B + B*)/2, and ≽ 0 means positive semidefinite. For ρ = 1 this reduces to the contraction condition ∥T∥ ≤ 1; for ρ = 2 it becomes I - Re(z̅T) ≽ 0 for every |z| ≤ 1, equivalent to numerical radius w(T) ≤ 1. Enforcing the inequality for a phase grid z = exp(iθ) gives a directly computable approximate Cρ certificate. The complete functional-calculus result additionally bounds matrix-valued analytic functions F evaluated at T, motivating stability tests for block- or channel-valued temporal filters.

## Key formulas

- $$T^{n}=\rho P_{\mathcal H}U^{n}|_{\mathcal H}\qquad\text{for all }n\geq 1,$$
- $$I-2\Big(1-\frac{1}{\rho}\Big)\operatorname{Re}(\overline{z}T)-\Big(\frac{2}{\rho}-1\Big)|z|^{2}T^{*}T\succeq 0\qquad\text{for all }z\in\overline{\mathbb D},$$
- $$\|T^n\|\leq \rho\qquad(n\geq 1),$$
- $$V_{\rho}(T)=\frac{1}{M}\sum_{j=0}^{M-1}\operatorname{softplus}\!\left(\tau\,\big[-\lambda_{\min}(I-2(1-\rho^{-1})\operatorname{Re}(e^{-i\theta_j}T)-(2/\rho-1)T^*T)\big]\right),\quad \theta_j=\frac{2\pi j}{M}.$$

## Implementation notes

(1) Integration point: replace the recurrent transition h_{t+1}=Th_t+Bx_t, or the SSM state matrix discretization, with a trainable complex or real matrix T. Apply the certificate as a training-time regularizer and optionally as a projection step after each optimizer update. Use ρ=1 for a strict contraction baseline, ρ=2 for a numerical-radius-constrained model, and test intermediate values such as 1.25 and 1.5. For real T, evaluate the complex phase condition using complex arithmetic or represent T as a real 2d×2d block matrix. (2) Pseudocode: initialize raw T; at each update construct T; for j=0,...,M-1 set z=exp(i*2πj/M), Q_j=I-2*(1-1/ρ)*Re(conj(z)*T)-(2/ρ-1)*abs(z)^2*T.H*T; compute v_j=-lambda_min(Q_j); add Vρ=(1/M)Σ softplus(τv_j) to the task loss; backpropagate and update T. Every K steps, optionally subtract η times the gradient of the largest violation. (3) Compute the smallest eigenvalues with 5–10 Lanczos iterations rather than a full eigendecomposition. The phase grid approximates the all-z condition, so validate with a denser grid and directly measure max_{1≤n≤N}∥T^n∥. The theorem predicts the target observable ∥T^n∥≤ρ, but does not require constructing the dilation U. (4) First experiment: compare a vanilla linear RNN, spectral-normalized linear RNN, and Cρ-regularized linear RNN on the adding problem and permuted sequential MNIST using equal hidden width and optimizer. Log training loss, gradient norm, hidden-state norm, maximum ∥T^n∥ for n=1,...,512, and validation accuracy. Success means no exploding trajectories, lower long-horizon gradient variance, and equal-or-better accuracy at sequence lengths where the unconstrained baseline becomes unstable; ρ>1 should preserve useful memory without uncontrolled growth.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.