# Symplectic Recurrent Block

- ID: 646
- Canonical URL: https://synthcore.org/idea/646/symplectic-recurrent-block
- API JSON: https://synthcore.org/api/idea/646.json
- API Markdown: https://synthcore.org/api/idea/646.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.09549](https://arxiv.org/abs/2607.09549)
- Category: architecture
- Solves: stability, accuracy
- ML areas: ssm, transformer, training-dynamics
- Math tags: symplectic, hamiltonian, dynamical-systems, numerical-analysis
- Ratings: usefulness 6/10; difficulty 5/10; novelty 5/10

## Idea description

Use a symplectic Hamiltonian update as a recurrent or state-space neural block, preserving a learned modified energy across many layers or time steps. This targets residual and recurrent architectures where ordinary Euler updates accumulate drift during long rollouts.

## Mathematical statement

For a phase-space state $y=(q,p)$, the paper's canonical Hamiltonian dynamics are $\dot y=J^{-1}\nabla H(y)$ with $J=\begin{bmatrix}0&-I\\I&0\end{bmatrix}$. A symplectic discretization is represented by backward error analysis as the exact flow of a modified vector field, $\Phi_h(y)=y+B_f(a)(y)=y+B_{\tilde f_h}(1/\gamma)(y)$, and for Hamiltonian systems $\tilde f_h=J^{-1}\nabla\tilde H_h$. Here $q,p$ are hidden-state position and momentum, $H_\phi$ is a learned scalar Hamiltonian, $h$ is the layer step, and $\phi$ are trainable weights. The transferable property is preservation of a nearby invariant over long compositions, not exact preservation of the original learned energy.

## Key formulas

- $$\dot{y}=X_H(y),\qquad X_H=J^{-1}\nabla H,\qquad J=\begin{bmatrix}0&-I\\ I&0\end{bmatrix}.$$
- $$\Phi_h(y)=y+B_f(a)(y)=y+B_{\tilde f_h}(1/\gamma)(y).$$
- $$\tilde f_h=J^{-1}\nabla\tilde H_h.$$
- $$p_{k+1/2}=p_k-\frac h2\nabla_q H_\phi(q_k,p_k),\quad q_{k+1}=q_k+h\nabla_pH_\phi(q_k,p_{k+1/2}),\quad p_{k+1}=p_{k+1/2}-\frac h2\nabla_qH_\phi(q_{k+1},p_{k+1/2}).$$

## Implementation notes

Insert the block between residual or state-space layers as a two-channel hidden state $(q,p)$, each with width $d$. Construct a scalar Hamiltonian $H_\phi(q,p,x)=U_\phi(q,x)+K_\phi(p)$, where $x$ is the block input, $U_\phi$ is a small MLP potential, and $K_\phi(p)=\frac12p^\top\operatorname{softplus}(m_\phi)^{-1}p$ with positive learned masses. Apply the three displayed leapfrog substeps, using automatic differentiation of the scalar Hamiltonian to obtain $\nabla_qH_\phi$ and $\nabla_pH_\phi$. Keep $x$ fixed during internal substeps. Use two or four substeps per block and expose either $q_{k+1}$ or a projected concatenation $[q_{k+1},p_{k+1}]$ to the next layer. Initialize $q$ from the incoming residual and $p=0$ or a learned projection of $x$. Train with the task loss; an optional small term $\lambda|H_{k+1}-H_k|^2$ may be used only as an ablation. The symplectic property follows analytically from the splitting, while the Hamiltonian scale and stable step size are learned or tuned. Test a six-layer residual MLP and a small Transformer on sequential MNIST, comparing Euler residual, gated residual, and symplectic blocks at matched parameters. Repeat each model 100–200 times at inference to test rollout stability. Record hidden-state norms, $H_k$, gradient norms, accuracy, and wall-clock cost. Success is bounded state and energy drift, fewer exploding gradients, and stable accuracy at long rollout lengths without disproportionate compute.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built a PyTorch learned separable-Hamiltonian SymplecticBlock with leapfrog substeps and a deterministic nonlinear oscillator verification. The core claim was clearly observed: leapfrog had Jacobian determinant 1.0, bounded norm, and small bounded energy error over 10,000 steps, while Euler had determinant 1.06352 and diverged rapidly. A tiny learned one-step regression also showed a slight MSE improvement, but with substantially more parameters, so the practical accuracy win is only a weak signal.

### Mechanism check

- Verdict: Built a PyTorch learned separable-Hamiltonian SymplecticBlock with leapfrog substeps and a deterministic nonlinear oscillator verification. The core claim was clearly observed: leapfrog had Jacobian determinant 1.0, bounded norm, and small bounded energy error over 10,000 steps, while Euler had determinant 1.06352 and diverged rapidly. A tiny learned one-step regression also showed a slight MSE improvement, but with substantially more parameters, so the practical accuracy win is only a weak signal.
- Confidence: 9/10
- Limitations: Only a toy nonlinear oscillator and tiny one-step regression were tested. No sequential-MNIST, Transformer, multilayer residual benchmark, matched-FLOP comparison, repeated training seeds, or GPU speed measurement was performed. The learned symplectic block is more expensive because it computes internal Hamiltonian gradients with autograd.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_experiment.py](https://synthcore.org/code/1425/bench_experiment.py)
- [bench_report.json](https://synthcore.org/code/1425/bench_report.json)
- [experiment.py](https://synthcore.org/code/1425/experiment.py)
- [report.md](https://synthcore.org/code/1425/report.md)
- [report_bench_2026-09-04T211541.md](https://synthcore.org/code/1425/report_bench_2026-09-04T211541.md)
- [results.json](https://synthcore.org/code/1425/results.json)
- [symplectic_block.py](https://synthcore.org/code/1425/symplectic_block.py)
- [Download all files as ZIP](https://synthcore.org/download/1425)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.