Optimal Nonergodic Primal-Dual Complexity of Efficient Inexact Parameter-Free Augmented Lagrangian Methods

What the math gives to ML

The paper develops an inexact augmented-Lagrangian architecture that reaches optimal nonergodic primal-dual complexity without proximal regularization, ergodic averaging, or prior knowledge of problem constants. The transferable asset is the combination of parameter-free accelerated inner solves with verifiable stationarity certificates and multiplier updates driven by the standard augmented Lagrangian. This suggests a constrained fine-tuning or structured-weight training optimizer that automatically increases inner accuracy and penalty strength using observable primal feasibility and stationarity residuals, rather than requiring a hand-tuned penalty schedule. The strongest initial test is on convex last-layer or adapter training with exact linear constraints, followed by a heuristic full-network experiment.