Idea description

Replace an unconstrained recurrent transition by a sequence of exact SU(1,1) hyperbolic updates. The layer processes each token with a 2-complex-dimensional state and preserves the indefinite energy |a|^2-|b|^2=1 exactly, preventing numerical drift while retaining non-unitary amplification and attenuation.

Formulas

$$\partial_x\binom{a}{b}=\begin{pmatrix}0&\overline{f(x)}e^{2\pi i x\xi}\\ f(x)e^{-2\pi i x\xi}&0\end{pmatrix}\binom{a}{b}$$

$$|a_f(\xi)|^2-|b_f(\xi)|^2=1$$

$$M_f^2=|f|^2I,\qquad \exp(hM_f)=\cosh(h|f|)I+\frac{\sinh(h|f|)}{|f|}M_f$$

$$\begin{aligned}a_{k+1}&=\cosh(hr_k)a_k+\frac{\sinh(hr_k)}{r_k}\,\overline{f_k}e^{2\pi i x_k\xi}b_k\\ b_{k+1}&=\cosh(hr_k)b_k+\frac{\sinh(hr_k)}{r_k}\,f_ke^{-2\pi i x_k\xi}a_k\end{aligned}$$

Mathematical statement

The paper evolves z(x,xi)=(a(x,xi),b(x,xi))^T by z'(x)=M_f(x,xi)z(x), where M_f(x,xi)=\begin{pmatrix}0&\overline{f(x)}e^{2\pi i x\xi}\\ f(x)e^{-2\pi i x\xi}&0\end{pmatrix}, f(x) is a complex-valued input potential, x is sequence position, and xi is a spectral or channel parameter. Since M_f lies in the Lie algebra su(1,1), the solution preserves the indefinite Hermitian form |a|^2-|b|^2=1, starting from (a,b)=(1,0). For a piecewise-constant step of width h with r=|f|, M_f^2=r^2I, so the exact local propagator is exp(hM_f)=cosh(hr)I+[sinh(hr)/r]M_f, with the r=0 value defined by continuity as I+hM_f. This propagator preserves |a|^2-|b|^2 exactly up to floating-point error and is the implementable structure-preserving update.

Implementation notes

Integrate this at the recurrent or SSM transition, not in the feed-forward projection. Let the input at position k be u_k in R^d. Use a learned linear map to produce d/2 complex potentials f_k and optionally use several fixed or learned frequencies xi_j, giving one state pair (a_{k,j},b_{k,j}) per frequency. Initialize every pair with a_{0,j}=1 and b_{0,j}=0. For each token and frequency, run the following update: r=abs(f); c=cosh(h*r); s=sinh(h*r)/max(r,epsilon); phase=exp(2*pi*i*x_k*xi_j); a_new=c*a+s*conj(f)*phase*b; b_new=c*b+s*f*conj(phase)*a. Concatenate real and imaginary parts of the terminal states, or expose log(abs(a)) and the phase as the recurrent output, then apply a learned output projection and residual connection. Compute c and s in float32 even under mixed precision; clip h*r to a safe range or use stable exp-based implementations for large values. The mathematical quantity computed exactly is the exponential of each local Lie-algebra generator; no matrix exponential or backpropagated ODE solver is needed. Empirically monitor the invariant error E_k=mean_j abs(abs(a_kj)^2-abs(b_kj)^2-1) and gradient norms. First test on sequential MNIST and the Long Range Arena copy and addition tasks using a 1-4 layer, 64-state model, comparing vanilla diagonal SSM, GRU, and unitary RNN at equal parameter count and FLOPs. Success means lower loss at long sequence lengths, substantially smaller invariant drift than Euler discretization, and fewer exploding or vanishing-gradient failures; also report throughput because the update is elementwise and parallel over frequencies.

Verification

Beats tuned baseline

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Significant win ✓

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Implemented the exact SU(1,1) hyperbolic recurrent update and numerically verified its core claim: M^2=|f|^2I and invariant drift stayed at 1.53e-14 over 1000 steps, versus 0.457 for Euler updates. On the small sequence task, SU(1,1) reached 0.875 validation accuracy versus 0.883 for a tanh RNN and was substantially slower, so structure-preserving stability was clearly observed but no practical learning win was demonstrated.

Agent confidence: 9/10

Baseline: Tanh RNN: validation loss 0.2591, accuracy 0.8828, 38 parameters, 9.17 s
Idea: SU(1,1): validation loss 0.2868, accuracy 0.8750, 26 parameters, 54.61 s; invariant max error 1.53e-14 versus Euler 0.4571

Limitations: Only a tiny 32-step synthetic sign-of-sum task was tested, with one seed and one small model; no sequential MNIST, Long Range Arena, equal-FLOP tuning, mixed-precision, long-horizon gradient study, or multi-seed statistical comparison was performed.

How to run: python3 su11_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.15895) Sep 2, 2026

Beats the tuned baseline by 9.3% (p=0.0081, wins 8 / 8 paired seeds; metric: lower is better).

Benchmark: Dynamics forecasting
Model: su11_scan_matched
Paired seeds: 8
Baseline mean: 0.035
Idea mean: 0.0318
Effect (Δ): -0.0032 (−9.3%; negative = idea better)
Wins: 8 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±3.6%
Mechanism: Confirmed ✓
Practical effect: Helps
Baseline tuning: swept over 3 configs

Limitations:

Only the registered built-in dynamics pendulum track was tested, using 800 training samples, 400 test samples, 8 epochs, one recurrent width, and three learning rates. Other tracks, larger-scale training, long-horizon gradients, mixed precision, and throughput were not tested.

How to run: python3 bench_su11.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 2, 2026 latest Worked ✓

Agent confidence: 8/10

The exact SU(1,1) exponential scan significantly outperformed the matched Euler discretization on the registered dynamics track. Mean test MSE was 0.0317771 versus 0.0350234, paired delta was -0.0032463, permutation p=0.0081, and the idea won all 8 paired seeds. The trained-model mechanism signature confirmed invariant preservation, with exact maximum drift 1.43e-6 versus 0.102 for Euler.

Baseline: Matched Euler discretization: mean test MSE 0.0350234136, std 0.0221484165, best lr 0.006, 193 parameters.
Idea: Exact SU(1,1) exponential: mean test MSE 0.0317771100, std 0.0208386502, best lr 0.006, 193 parameters; delta -0.0032463036, p=0.0081, 8/8 wins.

Benchmark result