Cubic-Rate Third-Order Langevin Optimizer

Integrate the method into the optimizer state update, replacing SGD with momentum or Adam's single velocity tensor by two fp32 tensors $v$ and $a$ matching the parameter shapes. For each minibatch, compute $g=\nabla_x\ell(x)$ and use it as an estimate of $\nabla U(x)$. With step size $dt$, damping $\gamma$, and temperature $\varepsilon$, use the following update: sample $\xi\sim\mathcal{N}(0,I)$; set $a\leftarrow a+dt(-g-\gamma a)+\sqrt{2\gamma\varepsilon dt}\,\xi$; set $v\leftarrow v+dt\,a$; set $x\leftarrow x+dt\,v$. Noise must enter only the acceleration state. Weight decay can be applied separately to $g$, and gradient clipping should be tested only as a controlled stability baseline. Every $K$ optimizer steps, estimate the most negative local curvature using 3--5 Hessian-vector products and a small Lanczos iteration. If the smallest estimated Hessian eigenvalue is $\lambda_{\min}$, define $\kappa=\max(0,-\lambda_{\min})$. Solve $r^3+\gamma r^2-\kappa=0$ by bisection on $[0,\kappa^{1/3}+\sqrt{\kappa/\gamma}+1]$, or use Newton iteration initialized at $\max(\kappa^{1/3},10^{-3})$. Enforce $dt\,r\leq0.2$ for explicit integration; reduce $dt$ when this bound is violated. A more ambitious controller changes $\gamma$ to target $dt\,r\approx0.1$. The SDE and cubic relation come from the paper; the curvature estimate and damping controller are empirical adaptations. Begin with a small MLP on a synthetic four-well objective or two-dimensional moons classification, then test a small ResNet on CIFAR-10. Match all methods by gradient evaluations and wall-clock time against SGD, SGD with momentum, and underdamped Langevin. Record loss descent, escape time from deliberately poor basins, parameter-norm growth, gradient variance, and test accuracy across at least 20 seeds. Success is faster escape or lower loss at equal compute without exploding $\|a\|$; failure is loss divergence, NaNs, or no improvement over momentum after tuning.