Subspace curvature-scaling high-index saddle dynamics for accelerating ill-conditioned saddle point searches

What the math gives to ML

The transferable asset is a curvature-scaled variant of high-index saddle dynamics: unstable Hessian eigenvectors already computed by a saddle solver are reused to build a low-rank inverse-curvature preconditioner. This removes the convergence-rate dependence on the smallest-magnitude negative eigenvalues, the main failure mode of reflected-gradient saddle searches near ill-conditioned saddles. A practical neural-network adaptation is a Hessian-subspace preconditioner for finding or traversing saddles in nonconvex loss landscapes, and potentially for stabilizing min-max training when a small set of adversarial directions dominates the dynamics. The first implementation should use Hessian-vector products and a small Lanczos subspace, with damped reciprocal Rayleigh curvatures to prevent numerical explosions.