Strong growth and Goldstein subgradients in piecewise smooth optimization

What the math gives to ML

The paper studies Goldstein subgradients, formed by convexifying gradients in a neighborhood rather than evaluating only the current nonsmooth gradient. Its transferable asset is a practical optimizer primitive for piecewise-smooth objectives: neighborhood gradient averaging can produce a smaller and more stable descent direction near activation boundaries, while geometrically shrinking the neighborhood is reported to produce approximately linear convergence on generic examples. The direct neural-network experiment is a sampled minimum-norm Goldstein update for ReLU or max-affine networks, combined with restarts and a geometric radius schedule.