Universal Clock Regularization for Recurrent Dynamics

Integrate this at the hidden-state update of a GRU, vanilla RNN, S4-like state-space model, or latent neural ODE. Add a small phase head \(\phi_\theta\) that outputs \((s,c)\), normalize it as \((s,c)/(s^2+c^2)^{1/2}\), and compute the angle with \(\operatorname{atan2}(s,c)\); unwrap angles only within each short training window. Pseudocode: roll out hidden states \(h_0,\ldots,h_T\); compute \(\phi_k=\operatorname{atan2}(s_k,c_k)\); form unwrapped differences \(d_k=\operatorname{unwrap}(\phi_{k+1}-\phi_k)\); set \(\hat\omega_k=d_k/\Delta t\); compute \(\bar\omega=\operatorname{mean}_k\hat\omega_k\); add \(\lambda_{clock}\operatorname{mean}_k[(\hat\omega_k-\bar\omega)^2]\) to the task loss; then backpropagate through the phase head and recurrent model. The paper supplies the phase equation and attractive-limit-cycle assumption; the phase encoder, finite-difference velocities, and variance are estimated from minibatch rollouts. Start with a cheap experiment on a GRU forecasting noisy Van der Pol and Lorenz-63 trajectories, comparing standard teacher-forced training against clock regularization at matched parameter counts. Evaluate free-running prediction for 10, 50, and 200 periods. The falsifiable prediction is that the clock model has lower normalized frequency error \(\mathcal{E}_{clock}\) and slower phase drift. During a parameter or learning-rate sweep, \(\mathcal{E}_{clock}\) should rise sharply before hidden-state norms or forecast error diverge. Sweep \(\lambda_{clock}\): insufficient regularization should leave high phase variance, whereas excessive regularization should suppress non-periodic behavior. Record the first value at which long-horizon phase error decreases without increasing one-step task loss by more than 5%.