# Gaussian-compensated Levy neural noise

- ID: 1499
- Canonical URL: https://synthcore.org/idea/1499/gaussian-compensated-levy-neural-noise
- API JSON: https://synthcore.org/api/idea/1499.json
- API Markdown: https://synthcore.org/api/idea/1499.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.27657](https://arxiv.org/abs/2607.27657)
- Category: sampling
- Solves: speedup, stability, sample-efficiency
- ML areas: diffusion-sampling, optimizer, training-dynamics
- Math tags: stochastic-processes, probability, numerical-analysis
- Ratings: usefulness 7/10; difficulty 5/10; novelty 8/10

## Idea description

Replace the unresolved small jumps of an infinite-activity stable Levy noise source in a neural SDE or stochastic optimizer with one Gaussian increment whose variance equals the discarded jump variance. Simulate only jumps above the cutoff exactly or by Poisson sampling, retaining the large-jump distribution while obtaining the paper's O(\varepsilon) Wasserstein error instead of the naive O(\varepsilon^{1-\alpha/2}) error.

## Mathematical statement

For stable-type intensity \(\lambda_\beta(u)=c u^{-1-\alpha}\), with amplitude \(u>0\), scale \(c>0\), and activity index \(0<\alpha<2\), the paper analyzes the small-amplitude region \((0,\varepsilon)\). For a centered small-jump integral \(R_\varepsilon=\int_{0<u<\varepsilon}u\,\widetilde N(\mathrm dt,\mathrm du)\), where \(\widetilde N\) is the compensated Poisson random measure, its variance per unit time is \(\sigma_\varepsilon^2=\int_0^\varepsilon u^2\lambda_\beta(u)\,\mathrm du=c\varepsilon^{2-\alpha}/(2-\alpha)\). The compensated approximation replaces \(R_\varepsilon\) by \(\sigma_\varepsilon Z\), with \(Z\sim\mathcal N(0,1)\), while retaining jumps with \(u\geq\varepsilon\). The stated theorem gives Wasserstein-1 error \(O(\varepsilon)\) for this replacement, versus \(O(\varepsilon^{1-\alpha/2})\) when all small jumps are simply discarded; the residual is asymptotically normal at rate \(O(\varepsilon^{\alpha/2})\). The associated jump-evaluation costs to reach tolerance \(\tau\) are \(O(\tau^{-\alpha})\) for compensation and \(O(\tau^{-2\alpha/(2-\alpha)})\) for naive truncation. If the neural SDE uses uncompensated positive jumps rather than a centered Levy integral, add the exact small-jump mean \(m_\varepsilon=\int_0^\varepsilon u\lambda_\beta(u)\,\mathrm du=c\varepsilon^{1-\alpha}/(1-\alpha)\) for \(\alpha<1\), or use the model's prescribed Levy drift convention for \(\alpha\geq1\).

## Key formulas

- $$\lambda_\beta(u)=c u^{-1-\alpha},\qquad 0<\alpha<2,$$
- $$\sigma_\varepsilon^2=\int_0^\varepsilon u^2\lambda_\beta(u)\,\mathrm du=\frac{c}{2-\alpha}\varepsilon^{2-\alpha},\qquad R_\varepsilon\approx \sigma_\varepsilon Z,\quad Z\sim\mathcal N(0,1),$$
- $$W_1(\text{naive},\text{exact})=O\!\left(\varepsilon^{1-\alpha/2}\right),\qquad W_1(\text{Gaussian-compensated},\text{exact})=O(\varepsilon),$$
- $$\text{cost}_{\mathrm{naive}}(\tau)=O\!\left(\tau^{-2\alpha/(2-\alpha)}\right),\qquad \text{cost}_{\mathrm{comp}}(\tau)=O(\tau^{-\alpha}).$$

## Implementation notes

Integrate this at the noise-generation line of a neural SDE solver, stochastic residual block, diffusion sampler with Levy perturbations, or optimizer that intentionally injects stable jump noise. For an Euler step of duration h, let the state be x and network drift be f_theta(x,t). Use the compensated Levy update:

```
inputs: x, t, h, epsilon, alpha, c
lambda_cut = c * epsilon**(-alpha) / alpha
K ~ Poisson(h * lambda_cut)
for j in 1..K:
    sample U_j from density alpha * epsilon**alpha * u**(-1-alpha), u >= epsilon
large = sum_j U_j
sigma2 = c * epsilon**(2-alpha) / (2-alpha)
small = sqrt(h * sigma2) * Normal(0,1)
x <- x + h*f_theta(x,t) + large + small
```

For a centered process, compensate the retained large jumps using the model's Levy drift convention; when the relevant first moment is finite, subtract `h * integral_epsilon^infinity(u*lambda(u) du)`. For an uncompensated positive-jump process with alpha<1, add the discarded small-jump mean `h*m_small`, where `m_small=c*epsilon**(1-alpha)/(1-alpha)`. The variance formula is computed analytically, while the Gaussian and Poisson variables are sampled normally. First run a 2-layer neural SDE or CIFAR-10 stochastic residual classifier with alpha in {0.5,1.0,1.5}. Compare exact-small-jump simulation at a tiny reference cutoff, naive truncation, and compensation at equal wall-clock time and equal large-jump evaluations. Measure terminal-state sliced-W1, validation accuracy, gradient variance, and loss versus runtime. Success is a log-log W1 slope near 1 for compensation, near 1-alpha/2 for naive truncation, and equal terminal-state accuracy with fewer jump evaluations or faster loss descent. Test alpha near 2, where the predicted cost advantage is largest.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: inconclusive
- Verdict: Built a reusable Gaussian-compensated Levy Euler update with Poisson-sampled large jumps, matched small-jump Gaussian variance, symmetric/one-sided modes, and the finite-mean correction for alpha<1. The toy sweep confirmed the predicted exponents across alpha=0.5, 1.0, and 1.5: variance 2-alpha, naive scale 1-alpha/2, compensated Wasserstein proxy 1, residual normality proxy alpha/2, and retained-jump rate -alpha. The mechanism is numerically supported, although no full neural-SDE training or direct finite-sample W1 comparison was established.

### Mechanism check

- Verdict: Built a reusable Gaussian-compensated Levy Euler update with Poisson-sampled large jumps, matched small-jump Gaussian variance, symmetric/one-sided modes, and the finite-mean correction for alpha<1. The toy sweep confirmed the predicted exponents across alpha=0.5, 1.0, and 1.5: variance 2-alpha, naive scale 1-alpha/2, compensated Wasserstein proxy 1, residual normality proxy alpha/2, and retained-jump rate -alpha. The mechanism is numerically supported, although no full neural-SDE training or direct finite-sample W1 comparison was established.
- Confidence: 9/10
- Limitations: The direct empirical W1 test was abandoned because symmetric stable samples have problematic or infinite first moments for alpha<=1 and heavy-tail saturation obscured asymptotic slopes. No neural network optimization, neural-SDE trajectory accuracy, wall-clock benchmark, GPU test, or exact reference simulation at a tiny lower cutoff was performed.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.18585
- Paired wins: 6/8
- Benchmark verdict: no significant win

## Artifacts

- [bench_levy.py](https://synthcore.org/code/584/bench_levy.py)
- [bench_report.json](https://synthcore.org/code/584/bench_report.json)
- [levy_compensation.py](https://synthcore.org/code/584/levy_compensation.py)
- [neural_levy_noise.py](https://synthcore.org/code/584/neural_levy_noise.py)
- [report.md](https://synthcore.org/code/584/report.md)
- [report_bench_2026-08-31T183619.md](https://synthcore.org/code/584/report_bench_2026-08-31T183619.md)
- [results.json](https://synthcore.org/code/584/results.json)
- [run_experiment.py](https://synthcore.org/code/584/run_experiment.py)
- [smoke_test.py](https://synthcore.org/code/584/smoke_test.py)
- [Download all files as ZIP](https://synthcore.org/download/584)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.