Parameter-Free Cubic-Regularized Newton Method: Sharp Complexity and Generalized Smoothness

What the math gives to ML

The paper's transferable asset is a parameter-free second-order optimization principle under the generalized third-order smoothness condition ||∇^3f(x)|| ≤ L0 + L1||∇f(x)||, where effective curvature variation grows with gradient magnitude. This suggests an adaptive cubic Newton optimizer that estimates its regularization strength online rather than using a globally tuned Hessian-Lipschitz constant. For neural networks, the realistic deployment is a low-dimensional or blockwise version using Hessian-vector products, trust-region-style acceptance tests, and larger validation batches to control minibatch noise. The main falsifiable benefit is faster loss reduction per expensive curvature computation and fewer unstable overshooting steps than Newton-CG, AdamW, or fixed-cubic baselines.