Piecewise smooth functions and conservative fields: calculus for nonsmooth nonconvex optimization beyond stratification
arXiv:2607.13973
2026
Optimization
1 ideas extracted · analyzed Aug 30, 2026
What the math gives to ML
The paper provides a conservative-field calculus for continuous piecewise-smooth functions without requiring semialgebraic structure or Whitney stratification. This is relevant to neural networks containing ReLU-like gates, clipping, masking, comparisons, routing, and custom branching, because it gives a pathwise chain rule for the gradients actually produced by automatic differentiation. The most practical transfer is an AD-aware stochastic optimizer that samples locally reachable branch gradients, averages them inside their conservative-field envelope, and monitors whether interface-induced gradient disagreement is destabilizing training.
Ideas from this paper
Unverified
2026
Replace the single arbitrary autodiff derivative at a piecewise-smooth interface with a sampled conservative-field gradient envelope. For each minibatch and parameter point, collect gradients from locally reachable branches, average them as a convex combination, and use the resulting direction in a stochastic update. This is intended for architectures with routing, clipping, hard masks, or custom continuous branching where ordinary autodiff can select an unstable branch.
Useful6/10
Difficulty6/10
Novelty6/10