Floquet-Stabilized Periodic Training Dynamics

Use a small residual MLP, RNN, or state-space model and modulate either the residual Jacobian or the optimizer update periodically. The simplest MVP is a periodically varying learning rate, $\eta_k=\eta_0[1+a\sin(2\pi k/K)]$, combined with periodically varying damping or weight decay, so local parameter perturbations are approximated by $A_k=-d_kI+J_k$, where $J_k$ is an empirical Hessian or local update Jacobian. A stronger test uses an RNN transition $h_{k+1}=\phi(W_kh_k+Ux_k)$ with $W_k=W_0+\epsilon W_1\sin(2\pi k/K)$. Estimate its period propagator by multiplying Jacobians $D_k=\partial h_{k+1}/\partial h_k$ over one cycle, $\Phi_K=D_{K-1}\cdots D_0$, and estimate $\rho(\Phi_K)$ with power iteration or several Jacobian-vector products. Pseudocode: compute phase; construct the modulated matrix or learning rate; execute forward and backward updates; once per period, propagate random tangent vectors through all $D_k$; estimate $\gamma_K=K^{-1}\log\rho(\Phi_K)$; reduce amplitude if $\gamma_K>0$. The theoretical quantities are the spectral-edge approximation and Floquet criterion; instantaneous eigenvalues, Jacobian products, and commutator contributions are estimated empirically. First experiment: train a width-256 residual MLP on CIFAR-10 using constant, sinusoidal, and random schedules matched for mean learning rate and weight decay. Sweep period $K$ and amplitude $a$. The prediction is a stability boundary where the measured adiabatic estimate $\bar\lambda$ crosses zero for large $K$, and potentially a high-frequency rescue interval where instantaneous Jacobians have positive rightmost eigenvalues but $\gamma_K<0$. Require the measured zero crossing to agree with $\bar\lambda=0$ within 20%, and compare gradient-norm decay, loss divergence, and validation accuracy across the predicted Floquet boundary.