Stability of Finite-Batch Particle Mean-Field Variational Inference Beyond Strong Convexity
arXiv:2608.11486
2026
Optimization
2 ideas extracted · analyzed Sep 1, 2026
What the math gives to ML
The paper provides a constructive way to quantify loss of contraction when the target potential is smooth but not strongly convex. Its key object, the curvature defect, converts negative or insufficient curvature into an explicit additive error rather than requiring a global strong-convexity assumption. This suggests curvature-defect-aware step-size control and stability monitoring for neural optimizers, especially in regions where Hessian curvature changes sign. The finite-batch decomposition is also useful experimentally because it separates errors caused by stochastic gradients, discretization, initialization, and intrinsic nonconvexity.
Ideas from this paper
Unverified
2026
Add a cheap directional curvature-defect estimator to an SGD or AdamW optimizer and shrink the step size only when the local gradient field loses the nominal contraction margin. Unlike a Hessian-norm trust-region rule, this directly measures the quantity that appears as additive instability in the Euler coupling estimate.
Useful6/10
Difficulty4/10
Novelty6/10
Unverified
2026
Train a small ensemble of parameter particles with stochastic gradients while penalizing excessive pairwise curvature defect. The ensemble acts as a low-cost variational or exploration population, and the defect penalty discourages particle pairs from entering strongly noncontractive regions without requiring the neural loss to be globally convex.
Useful5/10
Difficulty6/10
Novelty7/10