# Scrambled Sobol Diffusion Ensembles

- ID: 2108
- Canonical URL: https://synthcore.org/idea/2108/scrambled-sobol-diffusion-ensembles
- API JSON: https://synthcore.org/api/idea/2108.json
- API Markdown: https://synthcore.org/api/idea/2108.md
- Verification status: failed_benchmark
- Source: [arXiv:2608.11055](https://arxiv.org/abs/2608.11055)
- Category: sampling
- Solves: sample-efficiency, accuracy, speedup
- ML areas: diffusion, diffusion-sampling, batching
- Math tags: probability, numerical-analysis, measure-theory
- Ratings: usefulness 7/10; difficulty 6/10; novelty 7/10

## Idea description

Use Owen-scrambled Sobol points instead of independent Gaussian seeds for batched diffusion sampling, mapping each cube point through the component-wise inverse Gaussian CDF and the model's probability-flow ODE. Estimate ensemble expectations with importance weights computed from the target-to-proposal density ratio, so the estimator remains valid despite finite-step and learned-score transport errors.

## Mathematical statement

The paper starts from the target expectation $\mu=\mathbb{E}_{\mathbf{x}\sim\pi}[f(\mathbf{x})]=\int_{\mathbb{R}^{d}}f(\mathbf{x})\pi(\mathbf{x})\,\mathrm{d}\mathbf{x}$, where $\pi$ is the target density, $f$ is the metric, and $d$ is dimension. Let $u\in(0,1)^d$ be a scrambled-net point, let $z=\Phi^{-1}(u)$ componentwise with $\Phi$ the standard normal CDF and $\varphi(z)$ its standard normal density, and let $T_h$ be the $K$-step Euler approximation of the learned probability-flow transport. For a forward SDE with drift $b_t(x)$, scalar diffusion coefficient $\sigma_t$, and score $s_t(x)=\nabla_x\log p_t(x)$, the probability-flow vector field is $v_t(x)=b_t(x)-\tfrac12\sigma_t^2s_t(x)$; integrate $\dot{x}=v_t(x)$ in the direction taking the Gaussian prior to the data distribution. The induced proposal density is $q_h(x)=\varphi(z)/|\det J_{T_h}(z)|$ for $x=T_h(z)$, where $J_{T_h}$ is the transport Jacobian. Thus the cube integrand is $G(u)=f(T_h(\Phi^{-1}(u)))\pi(T_h(\Phi^{-1}(u)))|\det J_{T_h}(\Phi^{-1}(u))|/\varphi(\Phi^{-1}(u))$. The estimator is $\hat\mu_N=N^{-1}\sum_iG(u_i)$ over an Owen-scrambled Sobol net. Under the paper's boundary-growth and bounded-derivative assumptions, scrambled nets can achieve RMSE $O(N^{-1+\varepsilon})$ instead of ordinary Monte Carlo's $O(N^{-1/2})$. In applications without tractable $\pi$, first use the unweighted estimator and measure its bias against a large iid reference; in likelihood-known or toy settings, use the exact change-of-variables weight.

## Key formulas

- $$\mu\;=\;\mathbb{E}_{\mathbf{x}\sim\pi}[f(\mathbf{x})]\;=\;\int_{\mathbb{R}^{d}}f(\mathbf{x})\,\pi(\mathbf{x})\,\mathrm{d}\mathbf{x}.$$
- $$\frac{\mathrm{d}x_t}{\mathrm{d}t}=v_t(x_t),\qquad v_t(x)=b_t(x)-\frac{1}{2}\sigma_t^2\nabla_x\log p_t(x).$$
- $$z=\Phi^{-1}(u),\qquad x=T_h(z),\qquad q_h(x)=\frac{\varphi(z)}{|\det J_{T_h}(z)|},$$
- $$\widehat{\mu}_N=\frac{1}{N}\sum_{i=1}^{N}f(T_h(z_i))\frac{\pi(T_h(z_i))|\det J_{T_h}(z_i)|}{\varphi(z_i)},\qquad z_i=\Phi^{-1}(u_i),\quad u_i\sim\text{scrambled Sobol}.$$

## Implementation notes

Integrate this at the random-seed and inference-batching layer of a diffusion sampler. (1) Replace the usual iid tensor `eps = torch.randn(B,d)` with a precomputed Owen-scrambled Sobol tensor `U` of shape `(B,d)`; clamp values to `[1e-6,1-1e-6]`, transform with `Z=Phi^{-1}(U)`, and use `Z` as the terminal Gaussian state. (2) Run the deterministic probability-flow sampler: `x=Z; for k in reversed(range(K)): x = x + dt*(drift(x,t[k]) - 0.5*sigma(t[k])**2*score(x,t[k]))`, adapting the sign of `dt` to the implementation's time direction. (3) For a first MVP, compare unweighted Sobol outputs against iid Gaussian outputs for batch metrics. For a likelihood-known toy target, implement the correction by accumulating `logdet += dt*trace(J_v)`, where `J_v` is the Jacobian of the vector field. Estimate the trace with Hutchinson probes using `trace(J_v) approximately r^T J_v r`, with JVPs obtained by autodiff, and return `weight=exp(logdet + log_pi(x) - logvarphi(Z))`. The paper's mathematics determines the transport, change-of-variables correction, and RQMC estimator; the score, divergence, and Euler error are measured numerically. (4) First experiment: use a 2D Gaussian-mixture diffusion or CIFAR-10 DDIM with a small UNet. Evaluate batch means of pixel intensity, classifier score, and pairwise-distance statistics for batch sizes 64, 128, 256, and 512. Compare iid seeds, ordinary Sobol, and scrambled Sobol over at least 32 independent scramblings. Record randomization standard deviation, wall-clock cost, and bias relative to a 100,000-sample iid reference. Success means lower variance at equal denoising cost, ideally with an empirical slope approaching `N^-1` rather than `N^-1/2`; in the toy likelihood setting, the weighted estimator should additionally remove transport-approximation bias.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: no_effect
- Verdict: Built a self-contained toy scrambled-Sobol diffusion-transport MVP with exact change-of-variables weighting. The Jacobian check had absolute error 3.7e-11; observed unweighted bias matched the analytic quadratic prediction across lambda=0, 0.25, 0.5, 1.0, while weighting reduced bias at N=1024 to below 8e-5. Scrambled Sobol standard deviation scaled approximately N^-0.98 versus iid N^-0.43 at lambda=0.5, confirming the intended RQMC mechanism in this smooth toy setting.

### Mechanism check

- Verdict: Built a self-contained toy scrambled-Sobol diffusion-transport MVP with exact change-of-variables weighting. The Jacobian check had absolute error 3.7e-11; observed unweighted bias matched the analytic quadratic prediction across lambda=0, 0.25, 0.5, 1.0, while weighting reduced bias at N=1024 to below 8e-5. Scrambled Sobol standard deviation scaled approximately N^-0.98 versus iid N^-0.43 at lambda=0.5, confirming the intended RQMC mechanism in this smooth toy setting.
- Confidence: 9/10
- Limitations: Only a smooth 2D analytic transport surrogate was tested, not a learned neural diffusion score or high-dimensional image sampler. The Jacobian and importance weight were available analytically; Hutchinson trace estimation, finite-step ODE error with a learned model, GPU performance, and wall-clock batching costs were not evaluated.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 1
- Paired wins: 4/8
- Benchmark verdict: no measurable effect

## Artifacts

- [bench_report.json](https://synthcore.org/code/851/bench_report.json)
- [report.md](https://synthcore.org/code/851/report.md)
- [report_bench_2026-09-01T122610.md](https://synthcore.org/code/851/report_bench_2026-09-01T122610.md)
- [results.json](https://synthcore.org/code/851/results.json)
- [sobol_bench.py](https://synthcore.org/code/851/sobol_bench.py)
- [sobol_diffusion_mvp.py](https://synthcore.org/code/851/sobol_diffusion_mvp.py)
- [Download all files as ZIP](https://synthcore.org/download/851)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.