Curvature Residual Geometry in Bregman Regression

What the math gives to ML

The paper isolates a useful failure mode of Bregman regression: convexity of the scalar potential phi does not imply convexity of the parameterized objective because residuals enter the curvature coefficient. Its Hessian is a weighted Gram matrix, with weights determined jointly by phi'' at the prediction and phi''' multiplied by the residual. This gives a directly implementable curvature diagnostic for neural regression heads and a principled way to construct positive-semidefinite, residual-aware Gauss-Newton preconditioners rather than blindly applying the raw Hessian. The most promising transfer is a curvature-gated optimizer for nonquadratic regression or likelihood heads, where negative per-example curvature currently destabilizes second-order updates.