Unverified 2026

Cρ-stable recurrent transition

Implementation & benchmark of arXiv:2607.13794 — Complete functional calculus bounds for $ρ$-contractions

Usefulness6/10
Difficulty5/10
Novelty7/10

Source paper: Complete functional calculus bounds for $ρ$-contractions arXiv:2607.13794 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Constrain the transition matrix of an RNN or linear state-space model to the paper's class Cρ instead of controlling only its spectral radius or spectral norm. The resulting transition has an explicit dilation certificate and satisfies ∥T^n∥ ≤ ρ for every time horizon, preventing exploding hidden states while retaining nonnormal dynamics that ordinary spectral normalization may remove.

Formulas

$$T^{n}=\rho P_{\mathcal H}U^{n}|_{\mathcal H}\qquad\text{for all }n\geq 1,$$
$$I-2\Big(1-\frac{1}{\rho}\Big)\operatorname{Re}(\overline{z}T)-\Big(\frac{2}{\rho}-1\Big)|z|^{2}T^{*}T\succeq 0\qquad\text{for all }z\in\overline{\mathbb D},$$
$$\|T^n\|\leq \rho\qquad(n\geq 1),$$
$$V_{\rho}(T)=\frac{1}{M}\sum_{j=0}^{M-1}\operatorname{softplus}\!\left(\tau\,\big[-\lambda_{\min}(I-2(1-\rho^{-1})\operatorname{Re}(e^{-i\theta_j}T)-(2/\rho-1)T^*T)\big]\right),\quad \theta_j=\frac{2\pi j}{M}.$$

Mathematical statement

An operator T on a Hilbert space belongs to Cρ if there is a unitary U on a larger space K and an orthogonal projection P_H onto the model state space H such that T^n = ρ P_H U^n|_H for every integer n ≥ 1. Consequently, ∥T^n∥ ≤ ρ because ∥U^n∥ = 1 and ∥P_H∥ = 1. The paper gives the intrinsic certificate I - 2(1 - 1/ρ) Re(z̅T) - (2/ρ - 1)|z|^2 T*T ≽ 0 for every complex z with |z| ≤ 1, where I is the identity, T* is the conjugate transpose, Re(B) = (B + B*)/2, and ≽ 0 means positive semidefinite. For ρ = 1 this reduces to the contraction condition ∥T∥ ≤ 1; for ρ = 2 it becomes I - Re(z̅T) ≽ 0 for every |z| ≤ 1, equivalent to numerical radius w(T) ≤ 1. Enforcing the inequality for a phase grid z = exp(iθ) gives a directly computable approximate Cρ certificate. The complete functional-calculus result additionally bounds matrix-valued analytic functions F evaluated at T, motivating stability tests for block- or channel-valued temporal filters.

Implementation notes

(1) Integration point: replace the recurrent transition h_{t+1}=Th_t+Bx_t, or the SSM state matrix discretization, with a trainable complex or real matrix T. Apply the certificate as a training-time regularizer and optionally as a projection step after each optimizer update. Use ρ=1 for a strict contraction baseline, ρ=2 for a numerical-radius-constrained model, and test intermediate values such as 1.25 and 1.5. For real T, evaluate the complex phase condition using complex arithmetic or represent T as a real 2d×2d block matrix. (2) Pseudocode: initialize raw T; at each update construct T; for j=0,...,M-1 set z=exp(i*2πj/M), Q_j=I-2*(1-1/ρ)*Re(conj(z)*T)-(2/ρ-1)*abs(z)^2*T.H*T; compute v_j=-lambda_min(Q_j); add Vρ=(1/M)Σ softplus(τv_j) to the task loss; backpropagate and update T. Every K steps, optionally subtract η times the gradient of the largest violation. (3) Compute the smallest eigenvalues with 5–10 Lanczos iterations rather than a full eigendecomposition. The phase grid approximates the all-z condition, so validate with a denser grid and directly measure max_{1≤n≤N}∥T^n∥. The theorem predicts the target observable ∥T^n∥≤ρ, but does not require constructing the dilation U. (4) First experiment: compare a vanilla linear RNN, spectral-normalized linear RNN, and Cρ-regularized linear RNN on the adding problem and permuted sequential MNIST using equal hidden width and optimizer. Log training loss, gradient norm, hidden-state norm, maximum ∥T^n∥ for n=1,...,512, and validation accuracy. Success means no exploding trajectories, lower long-horizon gradient variance, and equal-or-better accuracy at sequence lengths where the unconstrained baseline becomes unstable; ρ>1 should preserve useful memory without uncontrolled growth.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.