Long-time Stability and Convergence of Particle Swarm Optimization

What the math gives to ML

The paper provides a discrete-time Schur-stability analysis for second-order stochastic optimization dynamics, including an explicit admissible parameter region that prevents oscillatory or divergent updates. The transferable asset is not PSO itself, but the conversion of a continuous damping and attraction model into a discrete stability constraint. A practical adaptation is a heavy-ball-like optimizer whose momentum and gradient step are chosen inside the paper's stability triangle, with curvature estimates used to keep the largest active eigendirection approximately stable. This is testable on small transformers and MLPs by comparing loss descent, gradient-norm spikes, and sensitivity to learning-rate increases against AdamW and momentum SGD.