# Wasserstein Poincare-deficit regularizer

- ID: 707
- Canonical URL: https://synthcore.org/idea/707/wasserstein-poincare-deficit-regularizer
- API JSON: https://synthcore.org/api/idea/707.json
- API Markdown: https://synthcore.org/api/idea/707.md
- Verification status: unverified
- Source: [arXiv:2607.10769](https://arxiv.org/abs/2607.10769)
- Category: regularization
- Solves: stability, generalization, accuracy
- ML areas: regularization, data-augmentation, mlp
- Math tags: optimal-transport, convex-analysis, probability, geometry
- Ratings: usefulness 6/10; difficulty 5/10; novelty 7/10

## Idea description

Construct intermediate training examples along an optimal-transport coupling between two strongly log-concave endpoint distributions, and regularize the network so that its output variance on each intermediate distribution is no larger than the sharp endpoint-interpolated Poincare scale times its expected input-Jacobian energy. This converts the paper's distributional inequality into a path-wise smoothness constraint for logits, embeddings, or scalar losses.

## Mathematical statement

For endpoint measures \(\mu_i(dx)=Z_i^{-1}e^{-V_i(x)}dx\), assume the potentials satisfy \(\nabla^2V_i(x)\succeq\kappa_i I\) with \(\kappa_i>0\). The Poincare constant is \(C_P(\mu)=\sup_f \operatorname{Var}_\mu(f)/\mathbb{E}_\mu\|\nabla f\|^2\), where \(\operatorname{Var}_\mu(f)=\mathbb{E}_\mu[(f-\mathbb{E}_\mu f)^2]\). If \(T=\nabla\phi\) is the Brenier map transporting \(\mu_0\) to \(\mu_1\), then \(\mu_t=((1-t)\operatorname{Id}+tT)_\#\mu_0\). The paper proves \(\sqrt{C_P(\mu_t)}\leq B_t\), where \(B_t=(1-t)\kappa_0^{-1/2}+t\kappa_1^{-1/2}\). Consequently, every smooth scalar network output \(f_\theta\) obeys \(\operatorname{Var}_{\mu_t}(f_\theta)\leq B_t^2\mathbb{E}_{\mu_t}\|\nabla_xf_\theta\|^2\). Use the empirical positive deficit \([\widehat{\operatorname{Var}}-B_t^2\widehat{\mathbb{E}}\|\nabla f\|^2]_+\) as a regularizer.

## Key formulas

- $$\sqrt{C_P(\mu_t)}\leq (1-t)\kappa_0^{-1/2}+t\kappa_1^{-1/2}=:B_t,$$
- $$\mu_t=\big((1-t)\operatorname{Id}+tT\big)_\#\mu_0,\qquad T=\nabla\phi,\quad T_\#\mu_0=\mu_1,$$
- $$\operatorname{Var}_{\mu_t}(f)\leq B_t^2\,\mathbb{E}_{x\sim\mu_t}\|\nabla_x f(x)\|_2^2,$$
- $$\mathcal{L}_{\mathrm{WP}}(\theta)=\frac{1}{|\mathcal{T}|}\sum_{t\in\mathcal{T}}\left[\widehat{\operatorname{Var}}_{b}(f_\theta(x_{t,b}))-B_t^2\frac{1}{B}\sum_{b=1}^{B}\|\nabla_x f_\theta(x_{t,b})\|_2^2\right]_+.$$

## Implementation notes

Integrate the method at the loss level for a classifier or encoder. Begin with two endpoint minibatches \(x_{0,b}\sim\mu_0\) and \(x_{1,b}\sim\mu_1\) with a known coupling. For a cheap controlled experiment, choose Gaussian endpoints \(\mu_i=\mathcal{N}(m_i,\Sigma_i)\). Their exact Brenier map is \(T(x)=m_1+A(x-m_0)\), with \(A=\Sigma_0^{-1/2}(\Sigma_0^{1/2}\Sigma_1\Sigma_0^{1/2})^{1/2}\Sigma_0^{-1/2}\). Compute \(x_{1,b}=T(x_{0,b})\), sample \(t\sim\operatorname{Uniform}(0,1)\), and set \(x_{t,b}=(1-t)x_{0,b}+tx_{1,b}\). For Gaussian endpoints use the exact curvature lower bound \(\kappa_i=1/\lambda_{\max}(\Sigma_i)\); for other distributions use a conservative analytic or estimated lower bound. Define the scalar output as the class margin \(f_\theta(x)=z_y(x)-\max_{k\neq y}z_k(x)\). Compute its minibatch variance and per-example input gradient with autodiff, then optimize `loss = cross_entropy + lambda * relu(var(f)-Bt**2*mean(sum(grad_x(f)**2)))`. Stop gradients through sampled transport points and \(B_t\). The theorem supplies the coefficient; minibatch variance, Jacobian energy, and transport quality are empirical. First compare against cross-entropy and ordinary input-gradient regularization on a two-dimensional Gaussian classification problem using a small MLP. Measure clean accuracy, accuracy on interpolated points, perturbation robustness, and the ratio \(\widehat{\operatorname{Var}}/(B_t^2\widehat{\mathbb{E}}\|\nabla f\|^2)\). A successful signal is a reduced ratio and improved transport-path robustness at equal FLOPs without clean-accuracy degradation. Then test the same loss on a frozen CIFAR-10 feature space with Gaussian endpoint clouds.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.