Difference-of-Convex Optimization via Inexact Smoothing Descent Methods: Difference of High-Order Moreau Envelopes

What the math gives to ML

The paper develops a constructive smoothing framework for difference-of-convex objectives using differences of high-order Moreau envelopes, together with inexact proximal solves and descent guarantees. The transferable asset is that a nonsmooth or unstable DC objective is replaced by a differentiable surrogate whose gradient is obtained from two proximal points, while the proximal subproblems may be solved only approximately. This suggests a neural optimizer for explicitly DC parameter objectives or DC local surrogates, especially when sparsity, capped penalties, pruning, or other nonconvex regularizers make direct minibatch gradients erratic. A continuation strategy is important because smoothing can create extra critical points: the paper's example shows one original stationary point expanding into an interval under smoothing.