Symplectic Hamiltonian Optimizer

Integrate this at the optimizer-update level. For parameters $\theta\in\mathbb R^d$, maintain momentum $p\in\mathbb R^d$ and define $H(\theta,p)=L(\theta)+\frac12p^\top M^{-1}p$, where $L$ is the minibatch loss and $M=\operatorname{diag}(m_i)$ is a positive mass vector. Use leapfrog: (1) compute $g=\nabla_\theta L(\theta)$ and set $p\leftarrow p-(h/2)g$; (2) set $\theta\leftarrow\theta+hM^{-1}p$; (3) recompute the gradient at the new parameters; (4) set $p\leftarrow p-(h/2)g$. This is a symplectic splitting of the canonical Hamiltonian dynamics. Compare against SGD with momentum and AdamW using the same number of gradient evaluations. Estimate the leading modified-energy behavior from pilot runs by recording $H_t$ and fitting $\Delta H_t=H_t-H_0$ against $h^p$ for $p\in\{1,2,3,4\}$. The mathematical structure is exact for the leapfrog map; the useful constants and stable step range must be estimated empirically. Initially omit weight decay, then test decay as a separate post-step operation. Use a two-layer MLP on MNIST or CIFAR-10, ten seeds, matched parameter count and gradient evaluations, and sweep $h$ and scalar masses. Measure loss versus optimizer steps, final validation accuracy, gradient norms, parameter oscillation, and divergence frequency. Success is lower long-horizon energy and gradient drift, fewer divergent runs, or better accuracy at equal compute.