Weak Information Geometry: Riemannian Structures from Distributional Inference Functions and Stein Discrepancies

What the math gives to ML

The paper replaces the Fisher metric, which requires a differentiable dominated likelihood, with a family of instrument-dependent Godambe metrics. The transferable asset is the explicit construction G(\theta)=S(\theta)^{\top}V(\theta)^{-1}S(\theta): sensitivity of chosen observables is weighted by their inverse variability, so unstable or redundant directions are automatically downweighted. This suggests a geometry-aware optimizer or adapter that preconditions neural-network updates using task-relevant probes rather than the full score-function Fisher matrix. The construction is especially attractive for heavy-tailed, implicit, or distribution-shifted data where likelihood-based natural-gradient methods are unavailable or poorly conditioned.