Wasserstein Poincare-deficit regularizer
Implementation & benchmark of arXiv:2607.10769 — Sharp Poincaré Interpolation Along Wasserstein Geodesics
Source paper: Sharp Poincaré Interpolation Along Wasserstein Geodesics arXiv:2607.10769 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Construct intermediate training examples along an optimal-transport coupling between two strongly log-concave endpoint distributions, and regularize the network so that its output variance on each intermediate distribution is no larger than the sharp endpoint-interpolated Poincare scale times its expected input-Jacobian energy. This converts the paper's distributional inequality into a path-wise smoothness constraint for logits, embeddings, or scalar losses.
Formulas
Mathematical statement
For endpoint measures \(\mu_i(dx)=Z_i^{-1}e^{-V_i(x)}dx\), assume the potentials satisfy \(\nabla^2V_i(x)\succeq\kappa_i I\) with \(\kappa_i>0\). The Poincare constant is \(C_P(\mu)=\sup_f \operatorname{Var}_\mu(f)/\mathbb{E}_\mu\|\nabla f\|^2\), where \(\operatorname{Var}_\mu(f)=\mathbb{E}_\mu[(f-\mathbb{E}_\mu f)^2]\). If \(T=\nabla\phi\) is the Brenier map transporting \(\mu_0\) to \(\mu_1\), then \(\mu_t=((1-t)\operatorname{Id}+tT)_\#\mu_0\). The paper proves \(\sqrt{C_P(\mu_t)}\leq B_t\), where \(B_t=(1-t)\kappa_0^{-1/2}+t\kappa_1^{-1/2}\). Consequently, every smooth scalar network output \(f_\theta\) obeys \(\operatorname{Var}_{\mu_t}(f_\theta)\leq B_t^2\mathbb{E}_{\mu_t}\|\nabla_xf_\theta\|^2\). Use the empirical positive deficit \([\widehat{\operatorname{Var}}-B_t^2\widehat{\mathbb{E}}\|\nabla f\|^2]_+\) as a regularizer.
Implementation notes
Integrate the method at the loss level for a classifier or encoder. Begin with two endpoint minibatches \(x_{0,b}\sim\mu_0\) and \(x_{1,b}\sim\mu_1\) with a known coupling. For a cheap controlled experiment, choose Gaussian endpoints \(\mu_i=\mathcal{N}(m_i,\Sigma_i)\). Their exact Brenier map is \(T(x)=m_1+A(x-m_0)\), with \(A=\Sigma_0^{-1/2}(\Sigma_0^{1/2}\Sigma_1\Sigma_0^{1/2})^{1/2}\Sigma_0^{-1/2}\). Compute \(x_{1,b}=T(x_{0,b})\), sample \(t\sim\operatorname{Uniform}(0,1)\), and set \(x_{t,b}=(1-t)x_{0,b}+tx_{1,b}\). For Gaussian endpoints use the exact curvature lower bound \(\kappa_i=1/\lambda_{\max}(\Sigma_i)\); for other distributions use a conservative analytic or estimated lower bound. Define the scalar output as the class margin \(f_\theta(x)=z_y(x)-\max_{k\neq y}z_k(x)\). Compute its minibatch variance and per-example input gradient with autodiff, then optimize loss = cross_entropy + lambda * relu(var(f)-Bt**2*mean(sum(grad_x(f)**2))). Stop gradients through sampled transport points and \(B_t\). The theorem supplies the coefficient; minibatch variance, Jacobian energy, and transport quality are empirical. First compare against cross-entropy and ordinary input-gradient regularization on a two-dimensional Gaussian classification problem using a small MLP. Measure clean accuracy, accuracy on interpolated points, perturbation robustness, and the ratio \(\widehat{\operatorname{Var}}/(B_t^2\widehat{\mathbb{E}}\|\nabla f\|^2)\). A successful signal is a reduced ratio and improved transport-path robustness at equal FLOPs without clean-accuracy degradation. Then test the same loss on a frozen CIFAR-10 feature space with Gaussian endpoint clouds.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.