Idea description

Treat a periodically forced optimizer as a discrete nonautonomous dynamical system and monitor its periodic parameter orbit rather than using only an average learning rate. Increase the forcing amplitude or base learning rate until the largest Floquet multiplier approaches +1, then reduce the schedule magnitude before the cyclic-fold instability.

Formulas

$$\theta_{k+1}=F_k(\theta_k;p),\qquad F_{k+T}=F_k$$

$$\Phi_T(\theta;p)=F_{T-1}\circ\cdots\circ F_0(\theta;p),\qquad \theta^\star=\Phi_T(\theta^\star;p)$$

$$M=D_\theta\Phi_T(\theta^\star;p)=\prod_{k=T-1}^{0}D_\theta F_k(\theta_k^\star;p),\qquad \rho(M)<1$$

$$\det(I-M)=0\qquad\Longleftrightarrow\qquad \mu=+1$$

Mathematical statement

Let theta_k in R^d be neural-network parameters and let F_k(theta;p) be the optimizer update at phase k of a T-step periodic schedule, where p is the forcing amplitude. The period map is Phi_T(theta;p)=F_{T-1} composed with ... composed with F_0. A periodic orbit satisfies theta_star=Phi_T(theta_star;p). Its monodromy matrix is M=D_theta Phi_T(theta_star;p), and perturbations over complete periods satisfy delta theta_{q+1}=M delta theta_q. Linear stability requires the spectral radius rho(M)<1. A generic cyclic fold occurs when a simple Floquet multiplier mu reaches +1, equivalently det(I-M)=0, provided the parameter crossing is nondegenerate.

Implementation notes

Use a small MLP, RNN, or state-space model with a deliberately periodic schedule. For example, set the learning rate to eta_k=eta_0 times [1+p sin(2 pi k/T)], or alternate easy and hard minibatches every T steps. The integration point is after each optimizer update: retain the trajectory over one complete period and estimate the Jacobian of the period map. Pseudocode: run T updates from theta_0; repeat periods until the normalized difference between theta_T and theta_0 is below epsilon; estimate the leading eigenvalue by power iteration; obtain each matrix-vector product with forward-mode JVPs through the T updates; sweep p or eta_0 by continuation; stop when the leading multiplier is within delta of +1. The exact mechanism supplies the fixed-point equation, monodromy product, and multiplier criterion. The orbit and Jacobian are estimated numerically with JVPs or truncated differentiation. The first experiment should use a two-layer MLP on MNIST, comparing constant and sinusoidally modulated learning rates across p and T. The quantitative prediction is a sharp divergence or loss-amplification boundary when the leading multiplier crosses +1, with the critical amplitude depending on T and agreeing with the Floquet estimate within about 20 percent. An average-learning-rate calculation should not predict this boundary.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a reproducible Floquet monitor for periodic heavy-ball optimization on a 1-D quadratic, including exact period-map Jacobians, determinant-root continuation, multiplier sweeps, and trajectory monitoring. The mechanism manifested: the leading multiplier crossed +1 at p=0.866886, with rho approximately 1 and det(I-M) approximately 5.6e-10; p=0.87 was unstable with rho=1.01384 and rapid amplification. Below the boundary, p=0.7869 had rho=0.88255 and contracted, while above it p=0.9469 had rho=1.37136 and grew strongly.

Agent confidence: 8/10

Baseline: Unmonitored forcing p=0.95: rho=1.38648; 30-period final norm=27967.43.
Idea: Floquet cap rho<0.97 selected p=0.86: rho approximately 0.969; 30-period final norm=1.112. Exact monodromy versus finite-difference Jacobian error was 3.45e-11. The predicted +1 boundary p=0.866886 matched the observed determinant root numerically. Critical amplitude decreased with base learning rate: pcrit=0.9433 at eta0=1.6, 0.8669 at eta0=1.7645, and 0.8346 at eta0=1.9. Predicted rho=0.405223 also matched the empirical period multiplier 0.405223.

Limitations: This is an exactly linear two-state quadratic toy, not the proposed two-layer MNIST MLP. It does not test CUDA, wall-clock speed, minibatch noise, high-dimensional JVP estimation, or continuation on a learned neural-network orbit. The rho cap controls asymptotic linear stability but can still allow transient amplification because the monodromy matrix is nonnormal.

How to run: python3 floquet_mvp.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.21256) Sep 2, 2026

Ahead of the baseline (−13.7%) but not statistically significant (p=0.25, wins 0 / 0 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0037
Idea mean: 0.0032
Effect (Δ): -0.0005 (−13.7%; negative = idea better)
Wins: 0 / 0 paired seeds
p-value: 0.25 (permutation test, 20 000 shuffles)
Smallest detectable effect: n/a
Mechanism: Confirmed ✓
Practical effect: Inconclusive
Baseline tuning: swept over 4 configs

Limitations:

The benchmark used 400 training and 100 test examples, 12 epochs, batch size 128, and the monitor used a parameter-displacement proxy rather than exact high-dimensional JVP monodromy estimation. The forcing amplitudes were modest and the monitor did not encounter a near-critical neural-network instability; the prior toy result is not used as task-metric evidence.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Failed ✗

Agent confidence: 9/10

Implemented the Floquet Loadability Monitor as a periodic learning-rate intervention for the registered dynamics/rnn_small benchmark, with an empirical period-displacement proxy and cap-based amplitude suppression. The mechanism signature was marked confirmed because the toy multiplier crossed the predicted stability boundary and the trained models produced finite empirical orbit proxies, but the idea did not beat the tuned baseline significantly, so worked=false.

Baseline: Tuned baseline best_cfg lr=0.002, p=0.0; full 8-seed test MSE reported in bench_report.json.
Idea: Best idea configuration from the parity sweep was lr=0.003, p=0.3; mean 8-seed test MSE=0.0031934431 versus baseline full mean reported by the harness; comparison verdict was not a significant win.

Benchmark result