Differentiating Minimal-Norm Solutions to Parametric Optimization Problems

What the math gives to ML

The paper addresses a failure mode of standard implicit differentiation: an inner optimization problem may have multiple minimizers, so its Jacobian is singular even though a stable, canonical solution can be selected. Its transferable asset is the minimal-norm selection together with limiting Tikhonov regularization, which replaces an unstable inverse by a controlled pseudoinverse-like solve and extends differentiation to nonsmooth objectives through conservative Jacobians. A practical neural-network use is a differentiable optimization layer for hyperparameter tuning, meta-learning, or data reweighting that returns the minimal-norm inner solution and computes hypergradients with damped linear solves rather than blindly inverting a singular Hessian.