Distributed Nonlinear Equality-Constrained Optimization via Feedback Linearization and Singular Perturbation

What the math gives to ML

The paper's transferable asset is a separation of constraint regulation from optimization along the feasible manifold: an ideal feedback-linearized direction exponentially reduces equality residuals, while a fast tracker approximates the otherwise nonlocal algebraic correction. For neural-network training, this suggests an optimizer that maintains differentiable equality constraints without relying on a large penalty coefficient, while retaining a tangent-space descent direction for the task loss. The most practical target is a blockwise constrained optimizer in which each parameter block has a small number of constraints, making the required Gram-matrix solves inexpensive.