# Universal Clock Regularization for Recurrent Dynamics

- ID: 961
- Canonical URL: https://synthcore.org/idea/961/universal-clock-regularization-for-recurrent-dynamics
- API JSON: https://synthcore.org/api/idea/961.json
- API Markdown: https://synthcore.org/api/idea/961.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.15472](https://arxiv.org/abs/2607.15472)
- Category: dynamics
- Solves: stability, accuracy, generalization
- ML areas: rnn, ssm, training-dynamics, regularization
- Math tags: dynamical-systems, control-theory, geometry, bifurcations, optimization
- Ratings: usefulness 7/10; difficulty 5/10; novelty 7/10

## Idea description

Add a learned phase coordinate to an RNN, state-space model, or latent neural ODE and train it to advance at constant angular velocity along recurrent trajectories. This separates genuine phase progression from amplitude and embedding distortions, encouraging coherent long-horizon oscillations while providing a quantitative monitor for impending loss of a limit cycle.

## Mathematical statement

For a continuous system \(\dot{\mathbf{x}}=\mathbf{F}(\mathbf{x})\), assume an attractive limit cycle \(\mathbf{C}\) and a phase function \(\phi(\mathbf{x})\in[0,2\pi)\) satisfying \(d\phi/dt=\omega\). By the chain rule, the defining clock equation is \(\nabla_{\mathbf{x}}\phi(\mathbf{x})\cdot\mathbf{F}(\mathbf{x})=\omega\), where \(\omega\) is the natural angular frequency. For a neural trajectory \(\mathbf{h}_k\) sampled at interval \(\Delta t\), use a phase encoder \(\phi_\theta(\mathbf{h})\) and estimate \(\widehat{\omega}_k=\operatorname{unwrap}(\phi_\theta(\mathbf{h}_{k+1})-\phi_\theta(\mathbf{h}_k))/\Delta t\). The adapted clock loss penalizes deviation from a trajectory-specific constant frequency: \(L_{\mathrm{clock}}=\operatorname{Var}_k(\widehat{\omega}_k)\), together with a cycle-rate loss matching the mean frequency to a learnable positive scalar \(\omega_\theta\). The property doing the work is low phase-velocity variance; its breakdown is predicted to precede amplitude or hidden-state divergence.

## Key formulas

- $$\frac{d\phi(\mathbf{x})}{dt}=\frac{\partial\phi}{\partial\mathbf{x}}\cdot\mathbf{F}(\mathbf{x})=\omega,$$
- $$\widehat{\omega}_k=\frac{\operatorname{unwrap}\!\left[\phi_\theta(\mathbf{h}_{k+1})-\phi_\theta(\mathbf{h}_{k})\right]}{\Delta t},\qquad L_{\mathrm{clock}}=\frac{1}{T-1}\sum_{k=0}^{T-2}(\widehat{\omega}_k-\overline{\omega})^2,$$
- $$L_{\mathrm{total}}=L_{\mathrm{task}}+\lambda_{\mathrm{clock}}L_{\mathrm{clock}}+\lambda_{\mathrm{cycle}}\left(\overline{\omega}-\operatorname{softplus}(a_\theta)\right)^2,$$
- $$\mathcal{E}_{\mathrm{clock}}=\frac{\sqrt{\operatorname{Var}_k(\widehat{\omega}_k)}}{|\overline{\omega}|+\epsilon},\qquad \overline{\omega}=\frac{1}{T-1}\sum_k\widehat{\omega}_k.$$

## Implementation notes

Integrate this at the hidden-state update of a GRU, vanilla RNN, S4-like state-space model, or latent neural ODE. Add a small phase head \(\phi_\theta\) that outputs \((s,c)\), normalize it as \((s,c)/(s^2+c^2)^{1/2}\), and compute the angle with \(\operatorname{atan2}(s,c)\); unwrap angles only within each short training window. Pseudocode: roll out hidden states \(h_0,\ldots,h_T\); compute \(\phi_k=\operatorname{atan2}(s_k,c_k)\); form unwrapped differences \(d_k=\operatorname{unwrap}(\phi_{k+1}-\phi_k)\); set \(\hat\omega_k=d_k/\Delta t\); compute \(\bar\omega=\operatorname{mean}_k\hat\omega_k\); add \(\lambda_{clock}\operatorname{mean}_k[(\hat\omega_k-\bar\omega)^2]\) to the task loss; then backpropagate through the phase head and recurrent model. The paper supplies the phase equation and attractive-limit-cycle assumption; the phase encoder, finite-difference velocities, and variance are estimated from minibatch rollouts. Start with a cheap experiment on a GRU forecasting noisy Van der Pol and Lorenz-63 trajectories, comparing standard teacher-forced training against clock regularization at matched parameter counts. Evaluate free-running prediction for 10, 50, and 200 periods. The falsifiable prediction is that the clock model has lower normalized frequency error \(\mathcal{E}_{clock}\) and slower phase drift. During a parameter or learning-rate sweep, \(\mathcal{E}_{clock}\) should rise sharply before hidden-state norms or forecast error diverge. Sweep \(\lambda_{clock}\): insufficient regularization should leave high phase variance, whereas excessive regularization should suppress non-periodic behavior. Record the first value at which long-horizon phase error decreases without increasing one-step task loss by more than 5%.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: inconclusive
- Verdict: Built a self-contained clock regularization MVP with a phase head, wrapped short-window phase differences, variance/cycle losses, analytical toy sweeps, and a fixed-seed GRU comparison. The core predictions manifested: phase-distortion variance and normalized clock error matched the derived scalings within 0.004% and 0.002%, noisy finite differences matched the 2σ²/Δt² law within 0.27%, and unwrap remained correct through increment 3.13 then switched branches above π as predicted. In the GRU toy, clock_E fell from 17.25 to 0.307 (98.2% lower), while one-step MSE worsened from 0.00519 to 0.00591 (13.8% higher), so this is a mechanism success and promising phase-consistency signal, not evidence of improved forecasting accuracy.

### Mechanism check

- Verdict: Built a self-contained clock regularization MVP with a phase head, wrapped short-window phase differences, variance/cycle losses, analytical toy sweeps, and a fixed-seed GRU comparison. The core predictions manifested: phase-distortion variance and normalized clock error matched the derived scalings within 0.004% and 0.002%, noisy finite differences matched the 2σ²/Δt² law within 0.27%, and unwrap remained correct through increment 3.13 then switched branches above π as predicted. In the GRU toy, clock_E fell from 17.25 to 0.307 (98.2% lower), while one-step MSE worsened from 0.00519 to 0.00591 (13.8% higher), so this is a mechanism success and promising phase-consistency signal, not evidence of improved forecasting accuracy.
- Confidence: 8/10
- Limitations: Only a short synthetic sinusoid GRU was tested; no Van der Pol, Lorenz-63, long-horizon free-running forecasts, λ sweep, limit-cycle breakdown prediction, matched-parameter phase-head ablation, or multi-seed confidence intervals were run. CUDA/cuDNN failed with CUDNN_STATUS_INTERNAL_ERROR_HOST_ALLOCATION_FAILED, so the reported GRU run used the implemented CPU fallback.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.5604
- Paired wins: 3/8
- Benchmark verdict: no significant win

## Artifacts

- [bench_clock.py](https://synthcore.org/code/404/bench_clock.py)
- [bench_report.json](https://synthcore.org/code/404/bench_report.json)
- [clock_experiment.py](https://synthcore.org/code/404/clock_experiment.py)
- [report.md](https://synthcore.org/code/404/report.md)
- [report_bench_2026-08-31T113806.md](https://synthcore.org/code/404/report_bench_2026-08-31T113806.md)
- [results.json](https://synthcore.org/code/404/results.json)
- [Download all files as ZIP](https://synthcore.org/download/404)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.