A Tractable Pseudo-Metric on Non-Parametric Exponential Statistical Manifolds via SPD Geometry

What the math gives to ML

The paper turns an intractable distributional geometry into a computable comparison by first matching finitely many sufficient-statistic moments and then representing the resulting exponential-family point by an SPD second-moment matrix. The transferable asset is the combination of a deliberately chosen moment map, an augmented outer-product embedding, and affine-invariant SPD distance, which avoids kernel bandwidths and is invariant to invertible linear reparameterizations. In neural networks this suggests comparing batches, feature distributions, or teacher-student activations through regularized SPD moment matrices rather than elementwise losses or manually tuned kernels. The construction is intentionally lossy: distributions with equal selected moments receive distance zero, so the sufficient statistics must be selected to match the target task.