A Local-Linearly Convergent Algorithm for Nonconvex Equality-Constrained Optimization

What the math gives to ML

The paper identifies a useful local regime for equality-constrained nonconvex optimization: the Gradient-Eigenstep method becomes ordinary gradient descent on Fletcher's augmented Lagrangian when the iterate is near a strong second-order stationary point, the step size is small, and the penalty is large. The transferable asset is the combination of warm starts, local linear contraction, and progressive sample enlargement for finite-sum objectives and constraints. A practical neural-network use is to train models subject to exact differentiable constraints by solving a sequence of increasingly large data subproblems, preserving the previous solution and switching to full-sample training only after local contraction appears.