Hamiltonian Horizon-Critical Optimizer

1. Integration point: use this as an optimizer wrapper around SGD or Adam on a small set of approximately independent parameter or curvature modes. Every K steps, obtain a diagonal or block-diagonal Hessian estimate \(\hat{\lambda}_i\) using one Hutchinson Hessian-vector product or a Lanczos pass. Represent the deviation of mode i from its local target by \(x_i\), set \(a_i=-\hat{\lambda}_i\), \(b_i=1\), \(q_i=\max(\hat{\lambda}_i,\epsilon)\), choose \(r_i\) from the squared gradient-noise scale, and set terminal penalty \(p_{f,i}=q_i\). 2. Pseudocode: estimate \(a_i,q_i,r_i\); construct \(H_i=[[a_i,-1/r_i],[-q_i,-a_i]]\); compute \(M_i=\exp(H_i T)\) for the proposed horizon T; compute \(D_i=M_{11}+M_{12}p_{f,i}\). If any \(D_i<\delta\), reduce T by multiplying it by 0.8 or increase \(r_i\) until all denominators exceed \(\delta\). Compute \(P_i=(M_{21}+M_{22}p_{f,i})/D_i\), then apply the controlled update \(\Delta\theta_i=-\eta_i(b_i/r_i)P_i x_i\), with the ordinary gradient direction used to define x_i or u_i in the full vector implementation. For a practical version, use the feedback multiplier \(g_i=(b_i/r_i)P_i\) on the Adam update. 3. Computed versus estimated: matrix exponentials and \(D_i\) are computed exactly for the estimated local quadratic model; \(a_i\), Hessian curvature, and noise-derived \(r_i\) are empirical estimates. Use clipping only as a safety fallback, not as the primary stabilization mechanism. 4. First cheap experiment: train a 2-layer MLP on MNIST or Fashion-MNIST with SGD, Adam, and this method, using a 20- to 100-dimensional Lanczos subspace rather than all parameters. Sweep T logarithmically and record loss, gradient norm, feedback gain, and \(\min_i D_i\). The quantitative prediction is a sharp optimizer transition when \(\min_i D_i\) approaches zero: gain variance and gradient excursions should rise sharply, while adaptive horizon control should prevent divergence. In the elliptic regime, the first singularity should occur near the zero of the analytically computed \(D_i(T)\), with measured instability threshold within 20%; away from the boundary, controlled training should exhibit lower loss oscillation than fixed-horizon SGD at matched update norm.