# Incompressible transport noise for feature maps

- ID: 2234
- Canonical URL: https://synthcore.org/idea/2234/incompressible-transport-noise-for-feature-maps
- API JSON: https://synthcore.org/api/idea/2234.json
- API Markdown: https://synthcore.org/api/idea/2234.md
- Verification status: unverified
- Source: [arXiv:2608.13332](https://arxiv.org/abs/2608.13332)
- Category: regularization
- Solves: stability, generalization, accuracy
- ML areas: cnn, regularization, training-dynamics
- Math tags: stochastic-processes, pde, differential-geometry
- Ratings: usefulness 6/10; difficulty 5/10; novelty 7/10

## Idea description

Replace independent additive noise on spatial feature maps with stochastic advection by divergence-free vector fields. The perturbation preserves spatial volume and feature mass, while the associated Stratonovich-to-Itô correction provides a tunable diffusion that preferentially damps high-frequency spatial fluctuations.

## Mathematical statement

The paper studies the Stratonovich stochastic transport term in equation (1.1), namely \(\sqrt{c_d\nu}\sum_{k\in\mathbb Z_0^d}\sum_{\alpha=1}^{d-1}\theta_k(\sigma_{k,\alpha}\cdot\nabla)v_i\circ dW_t^{k,\alpha}\), where \(v_i(t,x)\) is the \(i\)-th concentration or feature channel, \(\nu>0\) is the noise intensity, \(c_d=d/(d-1)\), \(\theta_k\) are mode weights, \(W_t^{k,\alpha}\) are independent Brownian motions, and \(\sigma_{k,\alpha}(x)\) are smooth divergence-free vector fields on the torus. Divergence-free means \(\nabla\cdot\sigma_{k,\alpha}=0\), so the flow transports features without local volume compression. Stratonovich-to-Itô conversion gives the deterministic second-order operator \(\frac{c_d\nu}{2}\sum_{k,\alpha}\theta_k^2(\sigma_{k,\alpha}\cdot\nabla)^2h\) for a scalar feature field \(h\). With an approximately isotropic collection of Fourier modes this behaves like effective diffusion \(\kappa\Delta h\), while finite-mode anisotropy can be measured from the covariance tensor. A periodic divergence-free basis is \(\sigma_{k,\alpha}(x)=e_{k,\alpha}\cos(2\pi k\cdot x)\), where \(e_{k,\alpha}\cdot k=0\); use \(d-1\) orthonormal vectors perpendicular to \(k\). Apply this operator independently to every feature channel, choosing \(\nu\) to control the desired spatial-frequency damping.

## Key formulas

- $$\mathrm{d}v_i-\nu_i\Delta v_i\,\mathrm{d}t=f_i(v)\,\mathrm{d}t+\sqrt{c_d\nu}\sum_{k\in\mathbb Z_0^d}\sum_{\alpha=1}^{d-1}\theta_k(\sigma_{k,\alpha}\cdot\nabla)v_i\circ\mathrm{d}W_t^{k,\alpha},\qquad c_d=\frac{d}{d-1}.$$
- $$\mathrm{d}h=\sqrt{c_d\nu}\sum_{k,\alpha}\theta_k(\sigma_{k,\alpha}\cdot\nabla)h\,\mathrm{d}W_t^{k,\alpha}+\frac{c_d\nu}{2}\sum_{k,\alpha}\theta_k^2(\sigma_{k,\alpha}\cdot\nabla)^2h\,\mathrm{d}t.$$
- $$\sigma_{k,\alpha}(x)=e_{k,\alpha}\cos(2\pi k\cdot x),\quad e_{k,\alpha}\cdot k=0,\quad \nabla\cdot\sigma_{k,\alpha}=0.$$
- $$\widehat h_{t+\Delta t}\approx\widehat h_t-\sqrt{c_d\nu}\sum_{k,\alpha}\theta_k\,\widehat{(\sigma_{k,\alpha}\cdot\nabla h_t)}\,\Delta W_{k,\alpha}+\frac{c_d\nu\Delta t}{2}\sum_{k,\alpha}\theta_k^2\widehat{(\sigma_{k,\alpha}\cdot\nabla)^2h_t}.$$

## Implementation notes

Integrate the method immediately before a spatial convolution or inside a residual block operating on a tensor H with shape [batch, channels, height, width]. Treat the two spatial axes as a periodic torus and do not transport across channels. Precompute integer wavevectors k with 1 <= |k| <= K and, for each k, perpendicular unit vectors e[k,alpha]. Construct divergence-free fields sigma[k,alpha](x)=e[k,alpha] cos(2 pi k dot x), optionally paired with sine fields. At each training step sample independent Gaussian increments dW[k,alpha]~N(0,dt). Use a midpoint or Heun discretization for the Stratonovich equation: compute G(H)=sum theta[k,alpha](sigma[k,alpha] dot grad)H; form H_pred=H+sqrt(c_d nu) G(H)dW; recompute G(H_pred); then set H_new=H+(sqrt(c_d nu)/2)(G(H)+G(H_pred))dW. Optionally add the explicit Ito correction D(H)=0.5*c_d*nu*sum theta[k,alpha]^2(sigma[k,alpha] dot grad)^2H using centered finite differences or spectral derivatives. Estimate the diffusion tensor A(x)=0.5*c_d*nu*sum theta[k,alpha]^2 sigma[k,alpha]sigma[k,alpha]^T and monitor its eigenvalue ratio as an anisotropy diagnostic. Start with a six-layer CIFAR-10 CNN and compare against Gaussian feature noise, dropout, and deterministic Gaussian blur at equal training FLOPs. Measure validation accuracy, corruption accuracy, feature-gradient norms, and the slope of high-frequency Fourier energy. Success means improved corruption robustness or generalization at unchanged clean accuracy, reduced high-frequency energy, and no increase in optimization instability. Ablate nu, mode count, mode frequency, sine/cosine pairing, and Euler versus midpoint discretization.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.