# Floquet Monodromy Optimizer

- ID: 1053
- Canonical URL: https://synthcore.org/idea/1053/floquet-monodromy-optimizer
- API JSON: https://synthcore.org/api/idea/1053.json
- API Markdown: https://synthcore.org/api/idea/1053.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.17072](https://arxiv.org/abs/2607.17072)
- Category: dynamics
- Solves: stability, speedup
- ML areas: optimizer, scheduler, training-dynamics
- Math tags: dynamical-systems, control-theory, linear-algebra, spectral-theory, numerical-analysis
- Ratings: usefulness 7/10; difficulty 5/10; novelty 7/10

## Idea description

Replace a stationary optimizer by a periodic two- or multi-phase schedule, such as alternating large and small learning rates, SGD and momentum, or gradients from different loss components. Stability is assessed over the complete period using the product of phase-wise linearized update maps, allowing a phase that is individually expansive to be safely combined with a contracting phase.

## Mathematical statement

For parameters \(\theta\in\mathbb{R}^d\), let phase \(i\) apply the differentiable update map \(F_i\) and let \(J_i=\partial F_i/\partial\theta\) be its local Jacobian near a reference trajectory. The one-period Floquet or monodromy matrix is \(M=J_k\cdots J_2J_1\). Repeated periods are locally stable when the spectral radius \(\rho(M)<1\), where \(\rho\) is the largest absolute eigenvalue. For continuous linearized phase dynamics \(\dot{\delta\theta}=A_i\delta\theta\), the effective Floquet generator is \(A_{\mathrm{eff}}=T^{-1}\log(\exp(A_kT_k)\cdots\exp(A_1T_1))\), with \(T=\sum_iT_i\). For two phases, the high-frequency Baker--Campbell--Hausdorff expansion contains the noncommutativity correction \([A_2,A_1]=A_2A_1-A_1A_2\), so phase order can change stability even when the time-averaged update is identical.

## Key formulas

- $$M=J_kJ_{k-1}\cdots J_1,\qquad \delta\theta_{n+1}=M\,\delta\theta_n,\qquad \text{stable iff }\rho(M)<1.$$
- $$J_i=\frac{\partial F_i(\theta)}{\partial\theta};\qquad F_i(\theta)=\theta-\eta_i g_i(\theta),\qquad J_i\approx I-\eta_iH_i,$$
- $$A_{\mathrm{eff}}=\frac{1}{T}\log\!\left(e^{A_2T_2}e^{A_1T_1}\right)=\frac{T_1A_1+T_2A_2}{T}+\frac{T_1T_2}{2T}[A_2,A_1]+O(T^2),\qquad T=T_1+T_2,$$
- $$\eta_{\mathrm{critical}}:\quad \rho\!\left[(I-\eta_2H_2)(I-\eta_1H_1)\right]=1.$$

## Implementation notes

1. Exact integration point: implement a periodic optimizer wrapper around a standard PyTorch optimizer. Divide each period into phase 1 and phase 2; for example, use SGD with learning rate \(\eta_1\) for \(m_1\) minibatches and SGD with learning rate \(\eta_2\) for \(m_2\) minibatches. A later variant can alternate SGD and AdamW or alternate gradients from two loss terms. Keep the average learning-rate budget fixed against a constant-SGD baseline.

2. Pseudocode:
```text
for period in range(P):
    for step in range(m1):
        loss = L(batch); g = grad(loss, theta)
        theta = theta - eta1 * g
    for step in range(m2):
        loss = L(batch); g = grad(loss, theta)
        theta = theta - eta2 * g
    if period % probe_interval == 0:
        estimate the Jacobian of the complete period using JVPs
        rho_hat = dominant_eigenvalue_or_singular_value(J_period)
        if rho_hat > 1: reduce eta1 or eta2
```
For a quadratic diagnostic problem, compute \(J_i=I-\eta_iH_i\) exactly and form \(M\). For a deep network, estimate the dominant singular value or eigenvalue using power iteration with Jacobian-vector products through the complete schedule period; use finite differences only for small models. The paper supplies the propagator-product and effective-generator construction, while Hessians, Jacobian-vector products, and spectral quantities are estimated empirically.

3. First cheap experiment: train a 2-layer MLP on MNIST with constant SGD versus alternating SGD schedules having equal average learning rate and periods \((m_1,m_2)\in\{(1,1),(5,5),(20,20)\}\). First use a quadratic regression surrogate with known Hessians \(H_1,H_2\), sweep \(\eta_1,\eta_2\), and compare the observed divergence boundary with \(\rho[(I-\eta_2H_2)(I-\eta_1H_1)]=1\). The predicted critical boundary should agree within 20%. In neural runs, perturb parameters slightly at the end of each period and measure perturbation growth. The quantitative prediction is decay when \(\hat\rho(M)<1\), growth when \(\hat\rho(M)>1\), and a sharp transition near one. Reverse the phase order; the performance and stability difference should increase with the estimated commutator norm \(\|J_2J_1-J_1J_2\|\).

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: no_effect
- Verdict: Built a reusable two-phase FloquetSGD optimizer and an exact 2D quadratic verification. The predicted stability boundary was scale=0.317659; perturbations decayed at rho=0.83099 and 0.98341, then grew at rho=1.01652 and 1.16213, confirming the monodromy criterion. The noncommutativity prediction matched exactly across the sweep, but the periodic schedule did not beat constant SGD on this quadratic, so the mechanism was verified without demonstrating an optimization speedup.

### Mechanism check

- Verdict: Built a reusable two-phase FloquetSGD optimizer and an exact 2D quadratic verification. The predicted stability boundary was scale=0.317659; perturbations decayed at rho=0.83099 and 0.98341, then grew at rho=1.01652 and 1.16213, confirming the monodromy criterion. The noncommutativity prediction matched exactly across the sweep, but the periodic schedule did not beat constant SGD on this quadratic, so the mechanism was verified without demonstrating an optimization speedup.
- Confidence: 9/10
- Limitations: Only a small exact quadratic diagnostic and an optimizer smoke test were run. No MNIST/MLP experiment, wall-clock benchmark, CUDA test, adaptive spectral-radius probing, AdamW phase, or noisy minibatch comparison was performed.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.7555
- Paired wins: 4/8
- Benchmark verdict: no measurable effect

## Artifacts

- [bench_floquet.py](https://synthcore.org/code/427/bench_floquet.py)
- [bench_report.json](https://synthcore.org/code/427/bench_report.json)
- [floquet_experiment.py](https://synthcore.org/code/427/floquet_experiment.py)
- [floquet_optimizer.py](https://synthcore.org/code/427/floquet_optimizer.py)
- [floquet_results.json](https://synthcore.org/code/427/floquet_results.json)
- [report.md](https://synthcore.org/code/427/report.md)
- [report_bench_2026-08-31T120443.md](https://synthcore.org/code/427/report_bench_2026-08-31T120443.md)
- [Download all files as ZIP](https://synthcore.org/download/427)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.