# Mollified Transport-Quantile Layer

- ID: 1137
- Canonical URL: https://synthcore.org/idea/1137/mollified-transport-quantile-layer
- API JSON: https://synthcore.org/api/idea/1137.json
- API Markdown: https://synthcore.org/api/idea/1137.md
- Verification status: unverified
- Source: [arXiv:2607.19080](https://arxiv.org/abs/2607.19080)
- Category: architecture
- Solves: stability, generalization, accuracy
- ML areas: embedding, vae, world-model, regularization
- Math tags: optimal-transport, geometry, statistics, pde
- Ratings: usefulness 6/10; difficulty 4/10; novelty 7/10

## Idea description

Use a transport map \(Q_\theta\) from a fixed latent reference distribution to a data distribution, but expose only its locally averaged version \(\bar Q_{\theta,\sigma}(z)=\mathbb E_{u\sim K_\sigma(\cdot-z)}Q_\theta(u)\). Latent-space mollification integrates the pole-type influence singularity instead of allowing one training sample near \(Q_\theta(z)\) to dominate the quantile feature or its gradient.

## Mathematical statement

The paper defines the transport quantile map as the unique squared-Euclidean optimal transport map \(\mathbf Q_P=T_{\mu\to P}\) satisfying \(\mathbf Q_P\#\mu=P\), where \(\mu\) is a fixed reference measure, \(P\) is the target distribution, and \(\#\) denotes push-forward. Under Huber contamination \(P_t=(1-t)P+t\delta_{x_0}\), the pointwise influence function is \(\mathbf I(x_0;\mathbf Q_P(z))=\lim_{t\downarrow0}[\mathbf Q_{P_t}(z)-\mathbf Q_P(z)]/t=\nabla G_{x_0}(z)\), where \(G_{x_0}\) solves the paper's uniformly elliptic Dirac-source problem with Neumann boundary conditions. For dimension \(d\ge2\), with \(\mathbf F_P=\mathbf Q_P^{-1}\), the paper establishes the pole behavior \(\|\mathbf I(x_0;\mathbf Q_P(z))\|\asymp\|z-\mathbf F_P(x_0)\|^{-(d-1)}\). The adapted layer convolves the query variable with a normalized kernel \(K_\sigma\): \(\bar{\mathbf Q}_{P,\sigma}(z)=\int K_\sigma(z-u)\mathbf Q_P(u)\,du\), \(K_\sigma(v)=\sigma^{-d}K(v/\sigma)\), \(\int K(v)dv=1\). The corresponding smoothed influence function is \(\bar{\mathbf I}_\sigma(x_0;z)=\int K_\sigma(z-u)\mathbf I(x_0;\mathbf Q_P(u))du\). Because the singularity is locally integrable in \(d\) dimensions, averaging over \(u\) removes the pointwise pole.

## Key formulas

- $$T_{\mu\to P}=\arg\min_{T:\,T\#\mu=P}\int_{\mathbb R^d}\|z-T(z)\|^2\,\mu(\mathrm dz),\qquad \mathbf Q_P=T_{\mu\to P},$$
- $$\mathbf I(x_0;\mathbf Q_P(z)):=\lim_{t\downarrow0}\frac{\mathbf Q_{(1-t)P+t\delta_{x_0}}(z)-\mathbf Q_P(z)}{t}=\nabla G_{x_0}(z),$$
- $$\|\mathbf I(x_0;\mathbf Q_P(z))\|\asymp\frac{1}{\|z-\mathbf F_P(x_0)\|^{d-1}},\qquad \mathbf F_P=\mathbf Q_P^{-1},\quad d\ge2,$$
- $$\bar{\mathbf Q}_{\theta,\sigma}(z)=\int K_\sigma(z-u)\mathbf Q_\theta(u)\,du\approx\frac1M\sum_{m=1}^M\mathbf Q_\theta(z+\sigma\varepsilon_m),\quad \varepsilon_m\sim K,\quad K_\sigma(v)=\sigma^{-d}K(v/\sigma).$$

## Implementation notes

1. Integration point: implement this as the readout of a transport decoder, quantile embedding, or VAE/world-model decoder whose network \(Q_\theta:\mathbb R^d\to\mathbb R^D\) maps fixed latent samples \(z\sim\mu\) to data-space points. Replace every pointwise call \(Q_\theta(z)\) used in a reconstruction loss, downstream predictor, or decoder output by the mollified call \(\bar Q_{\theta,\sigma}(z)\). Use Gaussian noise for \(K\), so \(\varepsilon_m\sim\mathcal N(0,I_d)\), and backpropagate through all \(M\) evaluations. 2. Pseudocode: inputs are query \(z\), bandwidth \(\sigma\), and Monte Carlo count \(M\); sample \(\varepsilon_1,\ldots,\varepsilon_M\); compute \(y_m=Q_\theta(z+\sigma\varepsilon_m)\); return \(\bar y=M^{-1}\sum_m y_m\). During training, draw fresh noise each step and anneal \(\sigma\) from an initial value such as \(0.1\) to a final value such as \(0.02\), or learn \(\log\sigma\) subject to a lower bound. Add a consistency penalty \(\|Q_\theta(z)-\bar Q_{\theta,\sigma}(z)\|^2\) if excessive smoothing reduces fidelity. 3. The paper supplies the transport definition, contamination derivative, and pole exponent \(d-1\); the implementation estimates the integral by Monte Carlo and treats \(\sigma\) and \(M\) as hyperparameters. Estimate empirical sensitivity using minibatch gradient variance \(\operatorname{tr}\operatorname{Cov}_B[\nabla_\theta\ell(\bar Q(z),x)]\), the 99.9th percentile of per-example gradient norms, and loss variance across repeated minibatches. 4. First experiment: train a small 2D transport decoder or VAE on a Gaussian-mixture dataset, comparing the pointwise baseline \(M=1,\sigma=0\) against \(M\in\{4,8\}\) and \(\sigma\in\{0.02,0.05,0.1\}\). Measure reconstruction error, held-out negative log likelihood or downstream classification accuracy, gradient-norm variance, maximum per-example gradient norm, and wall-clock cost. The hypothesis is a substantial reduction in gradient spikes and a smoother validation-loss curve, with equal or better accuracy; sweep \(M\) at inference to test whether \(M=4\) captures most of the stability benefit at modest cost.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.