Floquet Monodromy Optimizer

1. Exact integration point: implement a periodic optimizer wrapper around a standard PyTorch optimizer. Divide each period into phase 1 and phase 2; for example, use SGD with learning rate \(\eta_1\) for \(m_1\) minibatches and SGD with learning rate \(\eta_2\) for \(m_2\) minibatches. A later variant can alternate SGD and AdamW or alternate gradients from two loss terms. Keep the average learning-rate budget fixed against a constant-SGD baseline.

2. Pseudocode:

for period in range(P):
    for step in range(m1):
        loss = L(batch); g = grad(loss, theta)
        theta = theta - eta1 * g
    for step in range(m2):
        loss = L(batch); g = grad(loss, theta)
        theta = theta - eta2 * g
    if period % probe_interval == 0:
        estimate the Jacobian of the complete period using JVPs
        rho_hat = dominant_eigenvalue_or_singular_value(J_period)
        if rho_hat > 1: reduce eta1 or eta2

For a quadratic diagnostic problem, compute \(J_i=I-\eta_iH_i\) exactly and form \(M\). For a deep network, estimate the dominant singular value or eigenvalue using power iteration with Jacobian-vector products through the complete schedule period; use finite differences only for small models. The paper supplies the propagator-product and effective-generator construction, while Hessians, Jacobian-vector products, and spectral quantities are estimated empirically.

3. First cheap experiment: train a 2-layer MLP on MNIST with constant SGD versus alternating SGD schedules having equal average learning rate and periods \((m_1,m_2)\in\{(1,1),(5,5),(20,20)\}\). First use a quadratic regression surrogate with known Hessians \(H_1,H_2\), sweep \(\eta_1,\eta_2\), and compare the observed divergence boundary with \(\rho[(I-\eta_2H_2)(I-\eta_1H_1)]=1\). The predicted critical boundary should agree within 20%. In neural runs, perturb parameters slightly at the end of each period and measure perturbation growth. The quantitative prediction is decay when \(\hat\rho(M)<1\), growth when \(\hat\rho(M)>1\), and a sharp transition near one. Reverse the phase order; the performance and stability difference should increase with the estimated commutator norm \(\|J_2J_1-J_1J_2\|\).