Blockwise Stabilized Adaptive Cubic Regularization with Subsolvers via Recurrence

What the math gives to ML

The paper combines the Lipschitz-Hessian cubic upper bound with block-separable Newton steps, adaptive per-block regularization, and a monotone full-loss guard. The transferable asset is an optimizer that uses true Hessian-vector products rather than diagonal curvature while avoiding dense Hessian formation: each parameter tensor can be solved in its own Krylov subspace. Cubic regularization is especially attractive for neural losses because the model remains well-defined under negative curvature and can escape strict saddles, unlike a purely positive-definite preconditioner. The most practical transfer is a blockwise matrix-free cubic optimizer with backtracking on the cubic constant and rejection of steps that fail the actual-loss guard.