SynthCore
✗ Mechanism failed 2026

Pseudo-Hyperbolic Recurrent Dynamics

Implementation & benchmark of arXiv:2608.17021 — On Generalized Hyperbolicity, Stability, and Shadowing for Linear Operators

Usefulness8/10
Difficulty6/10
Novelty7/10
Dynamics StabilityAccuracyRnnSsmOptimizerTraining dynamicsDynamical systemsFunctional analysisSpectral theoryControl theory

Source paper: On Generalized Hyperbolicity, Stability, and Shadowing for Linear Operators arXiv:2608.17021 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace an unconstrained recurrent transition with two coupled channels: one contracts under forward iteration and the other contracts under inverse iteration. Enforcing this structure should prevent long-horizon amplification of state, numerical, and teacher-forcing perturbations while retaining nontrivial memory through the backward-stable channel.

Formulas

$$\|T^{n}Px\|\le C\beta^{n}\|x\|,\qquad \|T^{-n}(I-P)x\|\le C\beta^{n}\|x\|,\qquad n\ge 0,\quad 0<\beta<1.$$
$$A=Q\begin{bmatrix}A_s&0\\0&A_u\end{bmatrix}Q^{-1},\qquad \rho(A_s)\le\beta_s<1,\qquad \rho(A_u^{-1})\le\beta_u<1.$$
$$\mathcal{L}_{\mathrm{ph}}=\left[\log\frac{\widehat\rho(A_s)}{\beta_s}\right]_+^2+\left[\log\frac{\widehat\rho(A_u^{-1})}{\beta_u}\right]_+^2+\lambda_P\|P^2-P\|_F^2.$$
$$\sup_t\|\widetilde h_t-h_t^\star\|\lesssim\frac{K}{1-\beta}\sup_t\|\xi_t\|,\qquad \beta=\max(\beta_s,\beta_u).$$

Mathematical statement

For an invertible operator T and bounded projection P, pseudo-hyperbolicity requires constants C greater than zero and beta in (0,1) satisfying ||T^n P x|| <= C beta^n ||x|| and ||T^(-n)(I-P)x|| <= C beta^n ||x|| for every n >= 0. P selects the forward-stable subspace and I-P selects the backward-stable subspace. In a finite-dimensional recurrent layer, parameterize A in block form with a stable block A_s having spectral radius below one and an invertible block A_u whose inverse has spectral radius below one. Penalize violations of these two spectral conditions. The mechanism transferred from the paper is that pseudo-hyperbolicity implies structural stability and shadowing. For bounded additive perturbations with magnitude epsilon, the adapted recurrence predicts an error scale proportional to epsilon divided by 1 minus beta, where beta is the larger of the two contraction factors.

Implementation notes

1. Integration point: begin with a linear state-space model h_(t+1) = A h_t + B u_t, then extend to a nonlinear RNN by applying the same constraint to minibatch Jacobians. Use an orthogonal matrix Q and split the hidden state into z_s and z_u. Parameterize A_s with eigenvalues inside the unit disk, for example tanh(a_s), and A_u with eigenvalues outside the unit disk, for example sign(a_u) times (1 plus softplus(b_u)). 2. Pseudocode: initialize Q and the split; compute A_s and A_u; estimate rho_s = ||A_s^k||_2^(1/k) and rho_b = ||A_u^(-k)||_2^(1/k) with k equal to 4 or 8; add L_ph to the task loss; clip or reject updates if either estimate exceeds one minus a safety margin; then perform the recurrent rollout. For nonlinear cells, estimate products of Jacobian-vector products rather than forming full Jacobians. 3. Taken from the paper: the two exponential bounds and their implication for shadowing. Estimated empirically: the projection, contraction rates, constants C and K, and actual trajectory error. 4. First experiment: compare vanilla, spectral-normalized, and pseudo-hyperbolic RNNs on copy-memory and adding tasks with lengths 128, 512, and 2048. Inject bounded state noise with amplitudes from 10^-5 to 10^-2. Measure rollout error, gradient norms, and beta_hat. Prediction: for beta_hat below 0.9, error is approximately linear in noise amplitude with slope proportional to 1 divided by 1 minus beta_hat; near beta_hat equal to 1, error grows rapidly with horizon.

Verification

Mechanism failed

Stage 1 · Toy mechanism gate: Failed ✗

Stage 2 · Mechanism transferred to benchmark: Not tested

Stage 2 · Practical benchmark result: Not run

Methodology: Toy-system gate first; the benchmark stage runs only after a pass. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built a reproducible linear pseudo-hyperbolic dynamics MVP with projection/rate checks, noise-law sweeps, the q=1 amplification boundary, and a noisy recurrent comparison. The mathematical predictions matched numerically: contraction rates and epsilon/(1-beta) scaling were exact, and the transition from saturation to forward growth at q=1 was observed. However, the proposed split had mean final error 5.823e13 because its backward-stable channel expands under forward iteration, so it does not prevent causal forward perturbation amplification and gives no practical win here.

Agent confidence
9/10
Baseline
Stable spectral-normalized recurrence: mean final error 0.00406, p95 0.00583; unconstrained gain-1.03 recurrence: mean 0.2815, p95 0.4002.
Idea
Pseudo-hyperbolic split with gains 0.72 and 1.35: mean final error 5.823e13, p95 8.921e13 over 128 noisy forward steps. Rate checks matched 0.72 and 1/1.35=0.7407407 exactly; noise-law relative errors were <3e-15; q-boundary formula matched at q=0.8 through 1.1.

Limitations: No nonlinear trained RNN, copy-memory or addition task, teacher forcing, gradient measurements, optimization study, FLOP benchmark, or non-normal operator test was run. The toy construction uses diagonal matrices and tests forward causal noise, whereas the paper's shadowing result is fundamentally two-sided.

How to run: python3 experiment.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Artifacts

Implementation overview ⬇ Download all as ZIP 4 files · code, reports and structured results

Related ideas