# Convex-Projected Diffusion Sampler

- ID: 593
- Canonical URL: https://synthcore.org/idea/593/convex-projected-diffusion-sampler
- API JSON: https://synthcore.org/api/idea/593.json
- API Markdown: https://synthcore.org/api/idea/593.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.08757](https://arxiv.org/abs/2607.08757)
- Category: sampling
- Solves: stability, accuracy
- ML areas: diffusion, diffusion-sampling
- Math tags: convex-analysis, stochastic-processes, numerical-analysis
- Ratings: usefulness 8/10; difficulty 3/10; novelty 7/10

## Idea description

For additive-noise diffusion, train or interpret the network as a denoiser and project its predicted clean sample onto a known bounded closed convex set containing the data support. Convert the projected denoiser back into a score before each Euler-Maruyama or probability-flow ODE step. The projection is nonexpansive relative to the true denoiser, so it cannot increase pointwise denoising error when the true conditional mean belongs to the set, while it imposes a hard bound that suppresses rare numerical explosions.

## Mathematical statement

The paper studies the on-path score error $\mathcal{E}_{\mathrm{score}}=\int_{\delta}^{T}\mathbb{E}_{X_t\sim p_t}\|\widehat{s}_t(X_t)-s_t(X_t)\|^2\,dt$, where $p_t$ is the forward marginal, $s_t$ is the exact score, and $\widehat{s}_t$ is the learned score. This average error does not control the learned score on rare states visited by the discretized reverse process. For an additive Gaussian forward diffusion $X_t=X_0+\sigma_t\varepsilon$, define the clean-data denoiser $m_t(x)=\mathbb{E}[X_0\mid X_t=x]$ and the score relation $s_t(x)=(m_t(x)-x)/\sigma_t^2$, with noise scale $\sigma_t>0$. If the clean data satisfy $X_0\in K$ almost surely for a bounded closed convex set $K$, then $m_t(x)\in K$ because it is a conditional expectation of points in $K$. Let $\Pi_K$ be Euclidean projection onto $K$. The projection theorem gives $\|\Pi_K(a)-z\|\leq\|a-z\|$ for every $z\in K$, hence $\|\Pi_K(\widehat m_t(x))-m_t(x)\|\leq\|\widehat m_t(x)-m_t(x)\|$. It also gives $\|\Pi_K(\widehat m_t(x))\|\leq R$ when $K\subseteq\{z:\|z\|\leq R\}$, preventing arbitrarily large denoiser values. Use the adapted score $\widetilde s_t(x)=(\Pi_K(\widehat m_t(x))-x)/\sigma_t^2$ in the reverse sampler.

## Key formulas

- $$\mathcal{E}_{\mathrm{score}}=\int_{\delta}^{T}\mathbb{E}_{X_{t}\sim p_{t}}\|\widehat{s}_{t}(X_{t})-s_{t}(X_{t})\|^{2}\,dt.$$
- $$X_t=X_0+\sigma_t\varepsilon,\qquad m_t(x)=\mathbb{E}[X_0\mid X_t=x],\qquad s_t(x)=\frac{m_t(x)-x}{\sigma_t^2}.$$
- $$\|\Pi_K(a)-z\|\leq\|a-z\|\quad(z\in K),\qquad \widetilde m_t(x)=\Pi_K(\widehat m_t(x)),\qquad \widetilde s_t(x)=\frac{\widetilde m_t(x)-x}{\sigma_t^2}.$$
- $$x_{k-1}=x_k+h\,b_{\mathrm{rev}}(x_k,t_k;\widetilde s_{t_k})+g(t_k)\sqrt{h}\,\xi_k,\qquad \xi_k\sim\mathcal N(0,I).$$

## Implementation notes

Use a VE diffusion sampler first, because the denoiser-to-score relation is explicit. Let the network output $\widehat m_\theta(x,t)$, either by retraining the final head with denoising-score matching or by converting an existing score prediction using $\widehat m_\theta=x+\sigma_t^2\widehat s_\theta$. Choose a bounded closed convex set $K$: for normalized images, an elementwise box $[-1,1]^d$ is the cheapest baseline; for vector data use a ball or a tighter coordinate box estimated from the training set with a safety margin. At every reverse-time grid point, compute the denoiser, project it, convert it to a score, and pass that score to the existing reverse drift. Pseudocode: `x = sample_prior(); for k=T,...,1: m_hat = net(x,t[k]); m_proj = project_K(m_hat); s = (m_proj-x)/sigma(t[k])**2; x = x + h*reverse_drift(x,t[k],s) + g(t[k])*sqrt(h)*randn(); return x`. The mathematical guarantee is the projection inequality and the bounded-output property; estimate $K$, score calibration, and numerical effects empirically. First test on CIFAR-10 with a small DiT or U-Net, comparing unprojected Euler-Maruyama, projected Euler-Maruyama, and deterministic probability-flow sampling at identical networks and step counts. Track FID, precision/recall, maximum denoiser norm, fraction of trajectories exceeding a large state norm, endpoint $W_1$ against a reference fine-step sampler, and how these quantities change with step size. Success means substantially fewer rare large-norm trajectories and bounded endpoint moments, with no meaningful FID degradation; a stronger result is improved FID or Wasserstein error at coarse step counts.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built and ran a reproducible scalar VE/probability-flow Euler MVP for convex-projected denoising. All three mechanism predictions were confirmed: zero pointwise projection-error violations, an exact projected-denoiser bound, and monotone raw-tail instability with stable projected trajectories. This demonstrates the proposed mechanism, but not a general image-generation win.

### Mechanism check

- Verdict: Built and ran a reproducible scalar VE/probability-flow Euler MVP for convex-projected denoising. All three mechanism predictions were confirmed: zero pointwise projection-error violations, an exact projected-denoiser bound, and monotone raw-tail instability with stable projected trajectories. This demonstrates the proposed mechanism, but not a general image-generation win.
- Confidence: 9/10
- Limitations: Only a 1D controlled distribution and intentionally adversarial tail error were tested. No trained neural network, CIFAR/image metric, stochastic Euler-Maruyama comparison, runtime overhead, or misspecified convex-set experiment was evaluated.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [README.md](https://synthcore.org/code/117/README.md)
- [bench_report.json](https://synthcore.org/code/117/bench_report.json)
- [bench_run.py](https://synthcore.org/code/117/bench_run.py)
- [local_results.json](https://synthcore.org/code/117/local_results.json)
- [projected_denoiser_bench.py](https://synthcore.org/code/117/projected_denoiser_bench.py)
- [report.md](https://synthcore.org/code/117/report.md)
- [report_bench_2026-08-30T151645.md](https://synthcore.org/code/117/report_bench_2026-08-30T151645.md)
- [toy_results.json](https://synthcore.org/code/117/toy_results.json)
- [toy_verify.py](https://synthcore.org/code/117/toy_verify.py)
- [Download all files as ZIP](https://synthcore.org/download/117)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.