On The Linear Convergence of Bregman Proximal Gradient Methods with Applications to Kullback--Leibler regression

What the math gives to ML

The paper’s transferable contribution is a convergence framework for Bregman proximal gradient methods based on Restricted Relative Strong Convexity (RRSC), rather than ordinary Euclidean strong convexity. This suggests replacing fixed Euclidean updates with a domain-aware divergence for positive parameters, probability vectors, router probabilities, and normalized attention weights. The most practical first test is a smoothed Burg mirror optimizer whose smoothing prevents singular behavior near zero while retaining multiplicative-update geometry for KL-like or simplex-constrained objectives.