# Context-free denoiser with analytic quadratic score injection

- ID: 1156
- Canonical URL: https://synthcore.org/idea/1156/context-free-denoiser-with-analytic-quadratic-score-injectio
- API JSON: https://synthcore.org/api/idea/1156.json
- API Markdown: https://synthcore.org/api/idea/1156.md
- Verification status: beats_baseline
- Source: [arXiv:2607.19680](https://arxiv.org/abs/2607.19680)
- Category: sampling
- Solves: sample-efficiency, accuracy, scalability
- ML areas: diffusion, diffusion-sampling, world-model
- Math tags: probability, linear-algebra, stochastic-processes
- Ratings: usefulness 7/10; difficulty 6/10; novelty 6/10

## Idea description

Train one denoiser only for the nonquadratic residual distribution, then modify the diffusion sampler using an analytically computed quadratic Gaussian context. Changing $K$ at inference changes the target distribution without retraining the denoiser, enabling transfer across temperatures, masses, coupling strengths, and boundary conditions whenever those changes remain quadratic.

## Mathematical statement

The paper factors the target density as $\pi(\mathbf{x})\propto\exp[-\frac12\mathbf{x}^{\top}K\mathbf{x}]\exp[-U(\mathbf{x})]$, where $\mathbf{x}\in\mathbb{R}^{dNP}$ is the concatenated configuration over $P$ replicas, $K\succeq0$ is a known quadratic-context matrix, and $U(\mathbf{x})=\tau\sum_{k=1}^{P}V(\mathbf{x}_k)$ is the residual potential with $\tau=\beta/P$. The exact zero-noise score of this target is $\nabla_{\mathbf{x}}\log\pi(\mathbf{x})=-K\mathbf{x}-\nabla U(\mathbf{x})$. The transferable construction is to estimate the residual score $s_U(\mathbf{x})\approx-\nabla U(\mathbf{x})$ with a denoiser and compute the context contribution $s_K(\mathbf{x})=-K\mathbf{x}$ analytically. For Gaussian perturbation $y=x+\sigma\epsilon$, with $\epsilon\sim\mathcal{N}(0,I)$, a denoiser $D_\sigma(y)\approx\mathbb{E}[x\mid y]$ gives the residual noisy score $s_{U,\sigma}(y)=(D_\sigma(y)-y)/\sigma^2$ by Tweedie’s identity. The simple finite-noise composition $s_{U,\sigma}(y)-Ky$ is an approximation because Gaussian smoothing does not generally preserve products; the paper’s exactness claim applies when the training noise is matched to the intrinsic Gaussian uncertainty of the quadratic target. This approximation and the matched version should therefore be tested separately.

## Key formulas

- $$\pi(\mathbf{x})\propto\exp\!\Big[-\tfrac{1}{2}\mathbf{x}^{\top}K\mathbf{x}\Big]\exp\![-U(\mathbf{x})],\qquad U(\mathbf{x})=\tau\sum_{k=1}^{P}V(\mathbf{x}_k),\qquad \tau=\beta/P.$$
- $$\nabla_{\mathbf{x}}\log\pi(\mathbf{x})=-K\mathbf{x}-\nabla U(\mathbf{x}).$$
- $$s_{U,\sigma}(y)=\nabla_y\log p_{U,\sigma}(y)=\frac{D_\sigma(y)-y}{\sigma^2},\qquad D_\sigma(y)\approx\mathbb{E}[x\mid y],\quad y=x+\sigma\epsilon,\ \epsilon\sim\mathcal{N}(0,I).$$
- $$s_{K,\sigma}(y)\approx s_{U,\sigma}(y)-Ky,\qquad y\leftarrow y+\Delta t\,s_{K,\sigma}(y)+\sqrt{2\Delta t}\,\xi,\quad \xi\sim\mathcal{N}(0,I).$$

## Implementation notes

1. Integration point: modify the score evaluation inside a DDPM, score-SDE, or annealed Langevin sampler. Let the neural network receive the noisy state $y\in\mathbb{R}^{dNP}$ and noise level $\sigma$, but do not provide $K$ as an input. Train it on samples from the residual density $p_U(x)\propto\exp[-U(x)]$ with Gaussian corruption, using the denoising objective $\mathbb{E}\|D_\sigma(x+\sigma\epsilon)-x\|^2$. At sampling time, choose any valid positive-semidefinite context matrix $K$ and add its action analytically. 2. Pseudocode: `for sigma in noise_schedule: for step in range(n_steps[sigma]): eps=randn_like(y); D=denoiser(y,sigma); s_res=(D-y)/sigma**2; s=s_res-matvec(K,y); y=y+dt*s+sqrt(2*dt)*eps`. For a ring polymer, implement `matvec(K,y)` without materializing $K$: the cyclic finite-difference contribution at slice $k$ is proportional to $M(2y_k-y_{k-1}-y_{k+1})/(\hbar^2\tau)$, where $M$ is the particle mass matrix and periodic indexing is used. 3. Compute from the paper’s mathematics: the quadratic score $-Ky$, the eigenvalues of $K$, and the intrinsic uncertainty scale represented by the pseudoinverse $K^\dagger$; estimate the residual score empirically from the denoiser. If $K$ has a nullspace, project it out or use $K+\lambda I$ with a small ridge. Also implement a matched Gaussian variant: for each diffusion noise level, analytically propagate the quadratic Gaussian factor through the noise kernel and use its exact Gaussian score rather than simply $-Ky$. 4. First experiment: use a 2D or 4D double-well or coupled-quartic potential, train one small MLP denoiser on $p_U$, and compare (a) an unconditioned diffusion model, (b) a separately trained model for each $K$, and (c) one shared denoiser with analytic $K$ injection. Evaluate grid-reference KL divergence, energy histograms, covariance, replica correlations, and sampling time. Sweep temperature, mass, and cyclic versus open finite-difference matrices. Success means the shared model matches separately trained models at similar sample quality while requiring zero retraining and correctly changing covariance and path correlations.

## Verification

- Status: beats_baseline
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: helps
- Verdict: Built a residual double-well denoiser with inference-time quadratic score injection, including naive -Ky and matched Gaussian -k y/(1+k sigma^2) variants. The mechanism checks passed: predicted Gaussian score coefficients matched measured values to numerical precision (for example k=4,sigma=.7: 1.35135 vs 1.35135), and the Euler OU stability boundary appeared at dt·k=2.1 while dt·k=1.8 remained stable, with observed variances close to theory. In the toy sampler, analytic transfer was competitive with separately trained models (KL 0.373 vs 0.379 at k=2), but was worse at k=.5 (0.579 matched / 0.595 naive vs 0.566 separate), so this is a mechanism validation rather than a demonstrated universal quality win.

### Mechanism check

- Verdict: Built a residual double-well denoiser with inference-time quadratic score injection, including naive -Ky and matched Gaussian -k y/(1+k sigma^2) variants. The mechanism checks passed: predicted Gaussian score coefficients matched measured values to numerical precision (for example k=4,sigma=.7: 1.35135 vs 1.35135), and the Euler OU stability boundary appeared at dt·k=2.1 while dt·k=1.8 remained stable, with observed variances close to theory. In the toy sampler, analytic transfer was competitive with separately trained models (KL 0.373 vs 0.379 at k=2), but was worse at k=.5 (0.579 matched / 0.595 naive vs 0.566 separate), so this is a mechanism validation rather than a demonstrated universal quality win.
- Confidence: 8/10
- Limitations: Only a 2D double-well, one residual potential, two scalar quadratic contexts, one fixed noise-level sampling setup, and small MLPs were tested. No ring-polymer matrix-free implementation, nullspace handling, temperature/mass/boundary sweep, uncertainty calibration, FLOP-normalized training comparison, or high-dimensional diffusion schedule was evaluated; the reference and denoiser training chains are also approximate.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 8/8
- Benchmark verdict: idea better (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/461/bench_report.json)
- [context_bench.py](https://synthcore.org/code/461/context_bench.py)
- [experiment.py](https://synthcore.org/code/461/experiment.py)
- [quadratic_context_track.py](https://synthcore.org/code/461/quadratic_context_track.py)
- [report.md](https://synthcore.org/code/461/report.md)
- [report_bench_2026-08-31T124024.md](https://synthcore.org/code/461/report_bench_2026-08-31T124024.md)
- [results.json](https://synthcore.org/code/461/results.json)
- [Download all files as ZIP](https://synthcore.org/download/461)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.