# Pseudo-Hyperbolic Recurrent Dynamics

- ID: 2430
- Canonical URL: https://synthcore.org/idea/2430/pseudo-hyperbolic-recurrent-dynamics
- API JSON: https://synthcore.org/api/idea/2430.json
- API Markdown: https://synthcore.org/api/idea/2430.md
- Verification status: mechanism_failed
- Source: [arXiv:2608.17021](https://arxiv.org/abs/2608.17021)
- Category: dynamics
- Solves: stability, accuracy
- ML areas: rnn, ssm, optimizer, training-dynamics
- Math tags: dynamical-systems, functional-analysis, spectral-theory, control-theory
- Ratings: usefulness 8/10; difficulty 6/10; novelty 7/10

## Idea description

Replace an unconstrained recurrent transition with two coupled channels: one contracts under forward iteration and the other contracts under inverse iteration. Enforcing this structure should prevent long-horizon amplification of state, numerical, and teacher-forcing perturbations while retaining nontrivial memory through the backward-stable channel.

## Mathematical statement

For an invertible operator T and bounded projection P, pseudo-hyperbolicity requires constants C greater than zero and beta in (0,1) satisfying ||T^n P x|| <= C beta^n ||x|| and ||T^(-n)(I-P)x|| <= C beta^n ||x|| for every n >= 0. P selects the forward-stable subspace and I-P selects the backward-stable subspace. In a finite-dimensional recurrent layer, parameterize A in block form with a stable block A_s having spectral radius below one and an invertible block A_u whose inverse has spectral radius below one. Penalize violations of these two spectral conditions. The mechanism transferred from the paper is that pseudo-hyperbolicity implies structural stability and shadowing. For bounded additive perturbations with magnitude epsilon, the adapted recurrence predicts an error scale proportional to epsilon divided by 1 minus beta, where beta is the larger of the two contraction factors.

## Key formulas

- $$\|T^{n}Px\|\le C\beta^{n}\|x\|,\qquad \|T^{-n}(I-P)x\|\le C\beta^{n}\|x\|,\qquad n\ge 0,\quad 0<\beta<1.$$
- $$A=Q\begin{bmatrix}A_s&0\\0&A_u\end{bmatrix}Q^{-1},\qquad \rho(A_s)\le\beta_s<1,\qquad \rho(A_u^{-1})\le\beta_u<1.$$
- $$\mathcal{L}_{\mathrm{ph}}=\left[\log\frac{\widehat\rho(A_s)}{\beta_s}\right]_+^2+\left[\log\frac{\widehat\rho(A_u^{-1})}{\beta_u}\right]_+^2+\lambda_P\|P^2-P\|_F^2.$$
- $$\sup_t\|\widetilde h_t-h_t^\star\|\lesssim\frac{K}{1-\beta}\sup_t\|\xi_t\|,\qquad \beta=\max(\beta_s,\beta_u).$$

## Implementation notes

1. Integration point: begin with a linear state-space model h_(t+1) = A h_t + B u_t, then extend to a nonlinear RNN by applying the same constraint to minibatch Jacobians. Use an orthogonal matrix Q and split the hidden state into z_s and z_u. Parameterize A_s with eigenvalues inside the unit disk, for example tanh(a_s), and A_u with eigenvalues outside the unit disk, for example sign(a_u) times (1 plus softplus(b_u)). 2. Pseudocode: initialize Q and the split; compute A_s and A_u; estimate rho_s = ||A_s^k||_2^(1/k) and rho_b = ||A_u^(-k)||_2^(1/k) with k equal to 4 or 8; add L_ph to the task loss; clip or reject updates if either estimate exceeds one minus a safety margin; then perform the recurrent rollout. For nonlinear cells, estimate products of Jacobian-vector products rather than forming full Jacobians. 3. Taken from the paper: the two exponential bounds and their implication for shadowing. Estimated empirically: the projection, contraction rates, constants C and K, and actual trajectory error. 4. First experiment: compare vanilla, spectral-normalized, and pseudo-hyperbolic RNNs on copy-memory and adding tasks with lengths 128, 512, and 2048. Inject bounded state noise with amplitudes from 10^-5 to 10^-2. Measure rollout error, gradient norms, and beta_hat. Prediction: for beta_hat below 0.9, error is approximately linear in noise amplitude with slope proportional to 1 divided by 1 minus beta_hat; near beta_hat equal to 1, error grows rapidly with horizon.

## Verification

- Status: mechanism_failed
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built a reproducible linear pseudo-hyperbolic dynamics MVP with projection/rate checks, noise-law sweeps, the q=1 amplification boundary, and a noisy recurrent comparison. The mathematical predictions matched numerically: contraction rates and epsilon/(1-beta) scaling were exact, and the transition from saturation to forward growth at q=1 was observed. However, the proposed split had mean final error 5.823e13 because its backward-stable channel expands under forward iteration, so it does not prevent causal forward perturbation amplification and gives no practical win here.

### Mechanism check

- Verdict: Built a reproducible linear pseudo-hyperbolic dynamics MVP with projection/rate checks, noise-law sweeps, the q=1 amplification boundary, and a noisy recurrent comparison. The mathematical predictions matched numerically: contraction rates and epsilon/(1-beta) scaling were exact, and the transition from saturation to forward growth at q=1 was observed. However, the proposed split had mean final error 5.823e13 because its backward-stable channel expands under forward iteration, so it does not prevent causal forward perturbation amplification and gives no practical win here.
- Confidence: 9/10
- Limitations: No nonlinear trained RNN, copy-memory or addition task, teacher forcing, gradient measurements, optimization study, FLOP benchmark, or non-normal operator test was run. The toy construction uses diagonal matrices and tests forward causal noise, whereas the paper's shadowing result is fundamentally two-sided.

## Artifacts

- [REPORT.md](https://synthcore.org/code/813/REPORT.md)
- [experiment.py](https://synthcore.org/code/813/experiment.py)
- [report.md](https://synthcore.org/code/813/report.md)
- [results.json](https://synthcore.org/code/813/results.json)
- [Download all files as ZIP](https://synthcore.org/download/813)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.