# Fitted CFG terminal step

- ID: 554
- Canonical URL: https://synthcore.org/idea/554/fitted-cfg-terminal-step
- API JSON: https://synthcore.org/api/idea/554.json
- API Markdown: https://synthcore.org/api/idea/554.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.07665](https://arxiv.org/abs/2607.07665)
- Category: sampling
- Solves: stability, accuracy, speedup
- ML areas: diffusion-sampling, diffusion, inference-speedup
- Math tags: numerical-analysis, dynamical-systems, approximation-theory
- Ratings: usefulness 8/10; difficulty 3/10; novelty 8/10

## Idea description

Replace the standard CFG guidance multiplier in each DDIM step by a coefficient obtained from the terminal guided exponent. Given unconditional and conditional denoiser-derived states D_u and D_c, use r^{1+w}-r instead of w(r-1) on the guidance direction D_u-D_c; this preserves the same two denoiser calls and costs no additional NFE while suppressing low-noise residual blow-up at high guidance.

## Mathematical statement

The calibration model separates coordinates into shared-normal, tangential, and discriminative subspaces. In the discriminative subspace, classifier-free guidance changes the terminal scaling law from exponent 1 to exponent 1+w, where w>=0 is the CFG weight and sigma is the diffusion noise scale. For a step from sigma_n to sigma_{n+1}, define r=sigma_{n+1}/sigma_n, typically 0<r<1. Let x_n be the current sample, D_c the conditional denoiser prediction expressed as the DDIM terminal fitted point, and D_u the corresponding unconditional prediction. Standard CFG-DDIM is x_{n+1}=D_c+r(x_n-D_c)+w(r-1)(D_u-D_c). The paper's fitted operator instead integrates the guided discriminative exponent and gives x_{n+1}=D_c+r(x_n-D_c)+(r^{1+w}-r)(D_u-D_c). The scalar r^{1+w}-r is the unique spectrum-free coefficient that is exact on the discriminative terminal layer, agrees with CFG to first order as r approaches 1, reduces to ordinary DDIM when w=0, and avoids the sigma_min-divergent terminal residual predicted for coarse vanilla CFG steps. Here x_n, D_u, D_c are vectors in the latent or image state space; w is the guidance scale; sigma_n and sigma_{n+1} are consecutive noise levels; and r is computed directly from the sampler schedule.

## Key formulas

- $$x_{n+1}^{\mathrm{CFG}}=D_{\mathrm{c}}+r\,(x_n-D_{\mathrm{c}})+w(r-1)\,(D_{\mathrm{u}}-D_{\mathrm{c}}),\qquad r=\frac{\sigma_{n+1}}{\sigma_n}.$$
- $$x_{n+1}^{\mathrm{fitted}}=D_{\mathrm{c}}+r\,(x_n-D_{\mathrm{c}})+\bigl(r^{1+w}-r\bigr)(D_{\mathrm{u}}-D_{\mathrm{c}}).$$
- $$\text{guided terminal exponent in discriminative coordinates}=1+w,\qquad \text{unguided/DDIM exponent}=1.$$
- $$r^{1+w}-r\;\xrightarrow[w\to 0]{}\;0,\qquad r^{1+w}-r\;\approx\;w(r-1)\ \text{for }r\approx 1.$$

## Implementation notes

Integrate this at the deterministic DDIM or flow-matching sampler update, after both unconditional and conditional network evaluations have produced the two denoiser predictions. For an epsilon-prediction implementation, first convert each prediction to the same fitted-state parameterization used by the existing DDIM step; call these vectors D_u and D_c. For each schedule transition, compute r=sigma_next/sigma_current and replace only the guidance line. Pseudocode: `eps_u = model(x, sigma, null_condition); eps_c = model(x, sigma, condition); D_u = convert_to_ddim_state(x, eps_u, sigma); D_c = convert_to_ddim_state(x, eps_c, sigma); r = sigma_next / sigma; coeff = exp((1+w)*log(r)) - r; x_next = D_c + r*(x-D_c) + coeff*(D_u-D_c)`. Keep the same timestep schedule, model calls, prediction-type conversions, and precision as the baseline. For numerical robustness require positive sigma values and compute the power through `exp((1+w)*log(r))`; for very small w, compare against the baseline using a tolerance because cancellation can occur. The paper supplies the terminal exponent 1+w and the exact coefficient, so no spectrum or Jacobian needs to be estimated. Log the guidance-direction norm `||D_u-D_c||`, per-step residual norms, pixel or latent saturation statistics, and classifier-proxy accuracy. The first cheap experiment should use a pretrained CIFAR-10 diffusion model with 10, 20, and 50 DDIM steps and guidance weights 3, 5, 7.5, and 12, comparing vanilla CFG and fitted CFG at identical NFEs and seeds. Then run a small Stable Diffusion 1.5 prompt grid. Success means lower terminal residual amplification and fewer clipped or oversaturated samples, with equal or better FID at high guidance. Verify that w=0 exactly reproduces DDIM and that dense schedules approach vanilla CFG to first order.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: no_effect
- Verdict: Built a reproducible NumPy MVP for fitted CFG-DDIM updates and verified the proposed mechanism. The fitted coefficient is terminal-exact to 2.2e-17, w=0 reproduces DDIM within 6.7e-17, and the dense-schedule Taylor remainder has observed log-log slope 1.997 versus the predicted 2.0. On a coarse r=0.2 sweep, fitted guidance reduced coefficient magnitude by 80.0% to 97.9% for w=1 to 12, and the toy sampler terminal norm was 4.6% to 17.8% of vanilla CFG; this confirms stabilization, but it is not evidence of an image-quality win.

### Mechanism check

- Verdict: Built a reproducible NumPy MVP for fitted CFG-DDIM updates and verified the proposed mechanism. The fitted coefficient is terminal-exact to 2.2e-17, w=0 reproduces DDIM within 6.7e-17, and the dense-schedule Taylor remainder has observed log-log slope 1.997 versus the predicted 2.0. On a coarse r=0.2 sweep, fitted guidance reduced coefficient magnitude by 80.0% to 97.9% for w=1 to 12, and the toy sampler terminal norm was 4.6% to 17.8% of vanilla CFG; this confirms stabilization, but it is not evidence of an image-quality win.
- Confidence: 9/10
- Limitations: Only the analytical toy terminal-layer model and a synthetic equal-step vector sampler were tested. No pretrained CIFAR-10 or Stable Diffusion model, image metrics, classifier-proxy accuracy, clipping statistics, GPU execution, runtime comparison, or real denoiser prediction conversion was evaluated.

### Practical benchmark

- Paired seeds: 0
- Baseline mean: 0
- Idea mean: 0
- p-value: 0
- Paired wins: 0/0
- Benchmark verdict: blocked

## Artifacts

- [bench_report.json](https://synthcore.org/code/101/bench_report.json)
- [custom_diffusion_track.py](https://synthcore.org/code/101/custom_diffusion_track.py)
- [fitted_cfg_experiment.py](https://synthcore.org/code/101/fitted_cfg_experiment.py)
- [report.md](https://synthcore.org/code/101/report.md)
- [report_bench_2026-08-30T141125.md](https://synthcore.org/code/101/report_bench_2026-08-30T141125.md)
- [report_bench_2026-08-30T144312.md](https://synthcore.org/code/101/report_bench_2026-08-30T144312.md)
- [results.json](https://synthcore.org/code/101/results.json)
- [run_bench.py](https://synthcore.org/code/101/run_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/101)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.