Averaged Periodic Preconditioner

Implement this as an optimizer wrapper around a differentiable model. Choose $M$ positive diagonal preconditioners $P_0,\ldots,P_{M-1}$, or periodic learning-rate vectors, and apply $\theta\leftarrow\theta-\eta P_k g$ at phase $k$. Construct the averaged baseline $\bar P=M^{-1}\sum_{k=0}^{M-1}P_k$ and apply $\theta\leftarrow\theta-\eta\bar P g$ at each step. Keep the average update, batch order, and total gradient evaluations identical. The paper contributes the fast-coefficient averaging and contraction mechanism; strong convexity is only a local approximation in deep networks. Estimate local curvature with Hessian-vector products on a parameter block, estimate $p_{\min}$ from sampled Rayleigh quotients or the smallest diagonal preconditioner entry, and measure contraction by running two nearby parameter replicas with identical minibatches. The first experiment should use a two-layer MLP on MNIST or CIFAR-10, comparing SGD, AdamW, the periodic preconditioner, and its averaged version. Test cycle lengths $M\in\{1,2,4,8,16,32\}$ and synchronize trajectories after each cycle. The prediction is that, in a locally contractive phase, RMS parameter distance between periodic and averaged methods decreases as the physical oscillation period decreases, and replica distances have fitted decay slope near $-mp_{\min}$. When the measured local contraction ratio exceeds one, the approximation should cease to be reliable and trajectory error should begin growing.