# Koopman Skew-Dilation RNN

- ID: 2139
- Canonical URL: https://synthcore.org/idea/2139/koopman-skew-dilation-rnn
- API JSON: https://synthcore.org/api/idea/2139.json
- API Markdown: https://synthcore.org/api/idea/2139.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2608.11591](https://arxiv.org/abs/2608.11591)
- Category: dynamics
- Solves: stability, accuracy
- ML areas: rnn, ssm, training-dynamics, optimizer
- Math tags: dynamical-systems, control-theory, functional-analysis, linear-algebra, spectral-theory
- Ratings: usefulness 8/10; difficulty 6/10; novelty 6/10

## Idea description

Replace an unconstrained recurrent generator with a skew-adjoint block lift whose projected first channel implements a non-skew effective generator. The hidden state evolves unitarily in the enlarged space, preventing exponential norm blow-up, while the projection can express transient amplification, damping, and non-normal dynamics unavailable to a purely orthogonal recurrent matrix.

## Mathematical statement

For a Hilbert-space generator V, define its symmetric and antisymmetric parts S=(V+V*)/2 and A=(V-V*)/2, so S*=S, A*=-A, and V=S+A. The paper constructs a skew operator on the lifted space H_lift=ell^2(N) tensor H with diagonal blocks A, upper-diagonal blocks S, and lower-diagonal blocks -S. Therefore L*=-L and exp(tL) is unitary, implying norm preservation in the lifted space. Let P project onto the first channel and let E embed the original state into several channels. The embedding is chosen so that PLE x=(A+S)x=Vx for the first derivative. In a neural implementation, S is a learned symmetric matrix, A is a learned skew-symmetric matrix, L is the resulting block-tridiagonal lift, E is a fixed or learned channel embedding, and P is a decoder projection.

## Key formulas

- $$S=\frac{V+V^{*}}{2},\qquad A=\frac{V-V^{*}}{2},\qquad V=S+A,$$
- $$L=\begin{bmatrix}A&S&0&\cdots\\-S&A&S&\ddots\\0&-S&A&\ddots\\\vdots&\ddots&\ddots&\ddots\end{bmatrix},\qquad L^{*}=-L,$$
- $$\left(PLE\right)x=(A+S)x=Vx,\qquad \left\|e^{tL}z\right\|_{2}=\left\|z\right\|_{2},$$
- $$z_{k+1}=\left(I-\frac{\eta}{2}L\right)^{-1}\left(I+\frac{\eta}{2}L\right)z_k,\qquad h_k=Pz_k.$$

## Implementation notes

1. Integration point: implement this as a recurrent cell or linear latent state-space module. For hidden width d, use two to eight lifted channels, giving z in R^(md). Parameterize A=B-B^T and S=C+C^T, where B and C are trainable d by d matrices. Assemble the block-tridiagonal L with diagonal A, upper diagonal S, and lower diagonal -S. Use a fixed embedding E(x)=(x,x,0,...,0) and projection Pz=z^(1), or learn the projection only after validating the fixed version. 2. Pseudocode: construct L; form M=I-eta L/2 and N=I+eta L/2; solve M z_next=N z+E(u); output h=Pz_next; apply the task head to h. Use a dense solve for small models or a block-tridiagonal solver at larger widths. 3. Computed from the paper: the S/A decomposition, skew-adjoint block structure, unitary lifted flow, and projection identity. Estimated empirically: the projected Jacobian PLE, singular values, gradient norms, and approximation error to a target transition map. 4. First cheap experiment: train on sequential MNIST and a length-200 noisy linear-system prediction task, comparing vanilla RNN, orthogonal RNN, and the two-channel lift with matched parameter counts. Measure hidden norms, gradient norms, and prediction error as eta increases. The predicted signature is exact lifted norm preservation, ||z_k||_2=||z_0||_2 up to numerical tolerance for every eta under the Cayley update. Replacing Cayley by explicit Euler should produce norm growth approximately eta^2 ||Lz_k||_2^2 per step. The projected state may transiently amplify, but its gain is bounded by ||P|| ||E||.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built a finite-channel Koopman skew-dilation module with symmetric/skew parameter decomposition, block-tridiagonal lift, Cayley update, and a PyTorch toy sequence model. Quantitative predictions manifested: ||L+L^T||∞=0 and projection derivative error=1.1e-16; Cayley relative norm error stayed below 5.6e-16 for gamma∈{0,.25,.5,1,2} and eta∈{.05,.2,.5}; the explicit-Euler one-step norm increment exactly matched eta²||Lz||², while 10-step norm ratio grew from 1.04 at gamma=.25 to 12.95 at gamma=2. The toy task ended at MSE 0.0284 for vanilla RNN versus 0.000106 for the lift, but this is only a promising signal because parameter counts and architecture capacity were not strictly matched.

### Mechanism check

- Verdict: Built a finite-channel Koopman skew-dilation module with symmetric/skew parameter decomposition, block-tridiagonal lift, Cayley update, and a PyTorch toy sequence model. Quantitative predictions manifested: ||L+L^T||∞=0 and projection derivative error=1.1e-16; Cayley relative norm error stayed below 5.6e-16 for gamma∈{0,.25,.5,1,2} and eta∈{.05,.2,.5}; the explicit-Euler one-step norm increment exactly matched eta²||Lz||², while 10-step norm ratio grew from 1.04 at gamma=.25 to 12.95 at gamma=2. The toy task ended at MSE 0.0284 for vanilla RNN versus 0.000106 for the lift, but this is only a promising signal because parameter counts and architecture capacity were not strictly matched.
- Confidence: 9/10
- Limitations: No sequential MNIST experiment, no long-horizon benchmark, no wall-clock/FLOP comparison, and the toy lift has a larger latent state and more parameters than the baseline. The finite lift and dense linear solve do not test the infinite-dimensional theorem or scalable block-tridiagonal implementation.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/786/bench_report.json)
- [koopman_experiment.py](https://synthcore.org/code/786/koopman_experiment.py)
- [report.md](https://synthcore.org/code/786/report.md)
- [report_bench_2026-09-01T102800.md](https://synthcore.org/code/786/report_bench_2026-09-01T102800.md)
- [results.json](https://synthcore.org/code/786/results.json)
- [stage2_bench.py](https://synthcore.org/code/786/stage2_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/786)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.