# Hill-Floquet Regularization for Periodic RNNs

- ID: 1064
- Canonical URL: https://synthcore.org/idea/1064/hill-floquet-regularization-for-periodic-rnns
- API JSON: https://synthcore.org/api/idea/1064.json
- API Markdown: https://synthcore.org/api/idea/1064.md
- Verification status: mechanism_failed
- Source: [arXiv:2607.17339](https://arxiv.org/abs/2607.17339)
- Category: dynamics
- Solves: stability, accuracy
- ML areas: rnn, ssm, optimizer, training-dynamics
- Math tags: dynamical-systems, control-theory, spectral-theory, linear-algebra, numerical-analysis
- Ratings: usefulness 7/10; difficulty 6/10; novelty 8/10

## Idea description

Train a recurrent or state-space network together with a periodic hidden-state trajectory, then use the Fourier-domain Hill operator of its linearized dynamics to penalize positive Floquet growth rates. The method can retain algebraic hidden-state constraints, avoiding the inaccurate practice of treating a singular descriptor matrix as invertible.

## Mathematical statement

Let the hidden dynamics near a period-T orbit be the descriptor system A dot(delta x)(t) = J(t) delta x(t), where x(t) is the hidden state, A is a constant possibly singular descriptor matrix, and J(t) is the Jacobian along the periodic orbit. Write J(t) = sum_k J_k exp(i k omega t), with omega = 2 pi/T. For the perturbation delta x(t) = exp(mu t) sum_n p_n exp(i n omega t), the truncated generalized Hill problem is H_N(mu)p = 0, where p stacks Fourier coefficients, mu is a Floquet exponent, and the block entry is [H_N(mu)]_{n ell} = J_{n-ell} - (mu + i ell omega) A delta_{n ell}. Differential, constraint-consistent stability requires the largest real Floquet exponent to be negative; neutral phase modes may have zero real part. For singular A, algebraic modes must be removed with a consistency projector or Drazin inverse A^D rather than A^{-1}.

## Key formulas

- $$A\dot{\delta x}(t)=J(t)\delta x(t),\qquad J(t)=\sum_{k=-\infty}^{\infty}J_k e^{ik\omega t},\qquad \omega=\frac{2\pi}{T}.$$
- $$\left[H_N(\mu)\right]_{n\ell}=J_{n-\ell}-(\mu+i\ell\omega)A\,\delta_{n\ell},\qquad n,\ell\in\{-N,\ldots,N\},$$
- $$\mathcal{L}_{\mathrm{Hill}}=\operatorname{softplus}\!\left(\max_j\operatorname{Re}\mu_j+\varepsilon\right)^2,\qquad \rho_F=\exp\!\left(T\max_j\operatorname{Re}\mu_j\right).$$

## Implementation notes

Use this after each optimizer step, or every 100 steps, for an RNN or SSM trained on a periodically repeated sequence with period T. First obtain a periodic hidden trajectory x_*(t_m) by unrolling one period and requiring x_*(T) to approximately equal x_*(0), or by applying a short shooting correction. If the architecture has constrained states, express it as A dot(x) = f(t,x) with a singular diagonal mask A; for an unconstrained model use A = I. Sample M at least 4N+1 time points, compute J_m = partial f(t_m,x_*(t_m))/partial x with autodiff or Jacobian-vector products, and FFT every Jacobian entry to estimate J_k. Assemble H_N(mu), solve the truncated generalized eigenvalue problem, and obtain the dominant Floquet exponents. For singular A, discard eigenvectors with large algebraic-constraint residuals and use the Drazin-inverse or consistency projector when constructing tangent evolution. Add beta times L_Hill to the task loss; if eigenvalues are poorly separated, stop-gradient through the monitor and use it to reduce the learning rate. The paper supplies the Hill construction and stability criterion; the orbit, Fourier coefficients, Jacobians, and truncation error are estimated numerically. First experiment: compare a 32-unit tanh GRU and diagonal SSM on periodically forced sine prediction, with T = 2 pi, M = 128, N = 8, Adam baseline, and Adam plus the penalty. Independently propagate tangent vectors for one period to estimate the monodromy multiplier. The prediction is decay for rho_F < 1 and divergence for rho_F > 1, with the measured boundary within 20% of the Hill prediction and less than 10% change in the dominant exponent when N is increased from 4 to 8.

## Verification

- Status: mechanism_failed
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built a self-contained Fourier Hill-matrix MVP with independent monodromy integration and a toy regularization comparison. The exact offset-scaling prediction passed with slope 1.0, and true Floquet modes had Hill residuals below 6e-8, but the raw truncated Hill maximum produced a persistent spurious positive exponent and shifted the stability boundary by about 0.00816. Therefore the proposed practical largest-real-eigenvalue penalty was not reliable in this test and is not counted as a win.

### Mechanism check

- Verdict: Built a self-contained Fourier Hill-matrix MVP with independent monodromy integration and a toy regularization comparison. The exact offset-scaling prediction passed with slope 1.0, and true Floquet modes had Hill residuals below 6e-8, but the raw truncated Hill maximum produced a persistent spurious positive exponent and shifted the stability boundary by about 0.00816. Therefore the proposed practical largest-real-eigenvalue penalty was not reliable in this test and is not counted as a win.
- Confidence: 9/10
- Limitations: This is a 2D smooth linear toy rather than a trained GRU/SSM on sine prediction. Singular descriptor matrices, autodiff Jacobians, periodic-orbit shooting, GPU execution, and robust physical-mode continuation or structure-preserving Hill discretization were not tested.

## Artifacts

- [REPORT.md](https://synthcore.org/code/430/REPORT.md)
- [hill_floquet_experiment.py](https://synthcore.org/code/430/hill_floquet_experiment.py)
- [report.md](https://synthcore.org/code/430/report.md)
- [results.json](https://synthcore.org/code/430/results.json)
- [Download all files as ZIP](https://synthcore.org/download/430)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.