# Cyclic Momentum Latents

- ID: 785
- Canonical URL: https://synthcore.org/idea/785/cyclic-momentum-latents
- API JSON: https://synthcore.org/api/idea/785.json
- API Markdown: https://synthcore.org/api/idea/785.md
- Verification status: beats_baseline
- Source: [arXiv:2607.12143](https://arxiv.org/abs/2607.12143)
- Category: architecture
- Solves: stability, accuracy, generalization
- ML areas: world-model, vae, rnn, training-dynamics
- Math tags: dynamical-systems, geometry, optimization, linear-algebra
- Ratings: usefulness 7/10; difficulty 6/10; novelty 7/10

## Idea description

Add a learnable cyclic-coordinate mechanism to latent dynamics so that selected latent coordinates do not enter the Hamiltonian and their conjugate momenta become conserved. This provides an explicit dimensionality-discovery and invariance bias, encouraging the model to represent nuisance or symmetry directions compactly instead of spending independent dynamics capacity on them.

## Mathematical statement

Let $q_i$ be a candidate latent coordinate and $p_i$ its conjugate momentum. In Hamiltonian mechanics, if $H_\theta$ is independent of $q_i$, then $q_i$ is cyclic and $\dot p_i=-\partial H_\theta/\partial q_i=0$, so $p_i$ is conserved. Introduce a differentiable gate $m_i\in[0,1]$ indicating that coordinate $q_i$ should be cyclic, and penalize Hamiltonian dependence with $L_{cyc}=\sum_i m_i\mathbb{E}[(\partial H_\theta/\partial q_i)^2]$. To prevent every coordinate from becoming cyclic, add a budget term $\lambda_b(\sum_i m_i-r)^2$, where $r$ is the desired number of cyclic coordinates. The conservation diagnostic is $\delta p_i(T)=|p_i(T)-p_i(0)|$, which should remain near integration and optimization error for selected coordinates.

## Key formulas

- $$\dot p_i=-\frac{\partial H_\theta}{\partial q_i}=0\quad\text{when}\quad \frac{\partial H_\theta}{\partial q_i}=0.$$
- $$L_{cyc}=\sum_{i=1}^{d}m_i\,\mathbb{E}_{(q,p,c)}\left[\left(\frac{\partial H_\theta(q,p,c)}{\partial q_i}\right)^2\right],\qquad 0\le m_i\le1.$$
- $$L_{total}=L_{data}+\lambda_{cyc}L_{cyc}+\lambda_b\left(\sum_{i=1}^{d}m_i-r\right)^2.$$
- $$\delta p_i(T)=\left|p_i(T)-p_i(0)\right|\approx0\quad\text{for a genuinely cyclic coordinate}.$$

## Implementation notes

1. Integration point: attach a gate vector $m$ to the latent Hamiltonian of a world model, sequential VAE, or recurrent predictor. Use a sigmoid gate $m_i=\sigma(a_i/\tau)$ during training and anneal temperature $\tau$ so the representation becomes nearly discrete. Set a target number $r$ of cyclic coordinates, or use an $L_1$ penalty when the number is unknown. 2. Pseudocode: encode each trajectory into $(q,p)$; evaluate $H_\theta$; use autodiff to obtain every $\partial H/\partial q_i$; add $\lambda_{cyc}\sum_i m_i(\partial H/\partial q_i)^2$ to the prediction loss; propagate states with leapfrog; periodically report the variance and drift of each $p_i(t)$. Warm up with $\lambda_{cyc}=0$, then ramp it linearly after the predictor learns basic dynamics. 3. Computed from the mechanism: the cyclicity penalty follows directly from Hamilton's equation; gate values, momentum conservation, and effective latent dimension are estimated from data. The budget $r$ is a modeling choice. 4. First cheap experiment: use a synthetic oscillator with one active coordinate and one nuisance coordinate, then test a 4D latent model on pendulum trajectories under rotations or changed initial conditions. Compare a Hamiltonian model without gates, a fixed penalty on all coordinates, and the gated model. The prediction is a sharp separation in $\mathbb{E}[\delta p_i(T)]$: selected coordinates should have near-zero momentum drift, while noncyclic coordinates retain nonzero force gradients. Increasing $\lambda_{cyc}$ should produce a transition from dense to sparse coordinate usage; beyond the point where the true active coordinate is incorrectly gated, rollout error should increase sharply. Measure whether the discovered cyclic count matches $r$ and whether conservation error reaches the integrator's numerical-error floor.

## Verification

- Status: beats_baseline
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: helps
- Verdict: Built a reusable PyTorch gated Hamiltonian module with sigmoid cyclic gates, autodiff cyclicity and budget penalties, leapfrog integration, and momentum-drift diagnostics. The mechanism manifested: cyclic coordinates had zero measured momentum drift, the regularized optimum matched k*=k_true/(1+lambda*m) within 1.9e-14, and suppression disappeared at zero penalty or zero gate. A small noisy force-fitting comparison also reduced nuisance force magnitude 5x and test force MSE about 22x, though this is only a toy signal.

### Mechanism check

- Verdict: Built a reusable PyTorch gated Hamiltonian module with sigmoid cyclic gates, autodiff cyclicity and budget penalties, leapfrog integration, and momentum-drift diagnostics. The mechanism manifested: cyclic coordinates had zero measured momentum drift, the regularized optimum matched k*=k_true/(1+lambda*m) within 1.9e-14, and suppression disappeared at zero penalty or zero gate. A small noisy force-fitting comparison also reduced nuisance force magnitude 5x and test force MSE about 22x, though this is only a toy signal.
- Confidence: 8/10
- Limitations: Not tested on a learned encoder, sequential VAE, high-dimensional world model, pendulum trajectories, changed initial conditions, long-horizon rollout accuracy, learned gate discovery, or GPU training. The toy comparison uses a separable analytic Hamiltonian and a fixed nuisance gate rather than discovering gates from raw trajectories.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 8/8
- Benchmark verdict: idea better (significant)

## Artifacts

- [REPORT.md](https://synthcore.org/code/320/REPORT.md)
- [bench_cyclic_dynamics.py](https://synthcore.org/code/320/bench_cyclic_dynamics.py)
- [bench_report.json](https://synthcore.org/code/320/bench_report.json)
- [cyclic_latents.py](https://synthcore.org/code/320/cyclic_latents.py)
- [cyclic_momentum_experiment.py](https://synthcore.org/code/320/cyclic_momentum_experiment.py)
- [report.md](https://synthcore.org/code/320/report.md)
- [report_bench_2026-08-31T010737.md](https://synthcore.org/code/320/report_bench_2026-08-31T010737.md)
- [results.json](https://synthcore.org/code/320/results.json)
- [Download all files as ZIP](https://synthcore.org/download/320)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.