Fitted CFG terminal step
Implementation & benchmark of arXiv:2607.07665 — Guidance Breaks the Fitted Operator: A Terminal-Fitted Repair for Classifier-Free Guidance
Source paper: Guidance Breaks the Fitted Operator: A Terminal-Fitted Repair for Classifier-Free Guidance arXiv:2607.07665 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace the standard CFG guidance multiplier in each DDIM step by a coefficient obtained from the terminal guided exponent. Given unconditional and conditional denoiser-derived states D_u and D_c, use r^{1+w}-r instead of w(r-1) on the guidance direction D_u-D_c; this preserves the same two denoiser calls and costs no additional NFE while suppressing low-noise residual blow-up at high guidance.
Formulas
Mathematical statement
The calibration model separates coordinates into shared-normal, tangential, and discriminative subspaces. In the discriminative subspace, classifier-free guidance changes the terminal scaling law from exponent 1 to exponent 1+w, where w>=0 is the CFG weight and sigma is the diffusion noise scale. For a step from sigma_n to sigma_{n+1}, define r=sigma_{n+1}/sigma_n, typically 0<r<1. Let x_n be the current sample, D_c the conditional denoiser prediction expressed as the DDIM terminal fitted point, and D_u the corresponding unconditional prediction. Standard CFG-DDIM is x_{n+1}=D_c+r(x_n-D_c)+w(r-1)(D_u-D_c). The paper's fitted operator instead integrates the guided discriminative exponent and gives x_{n+1}=D_c+r(x_n-D_c)+(r^{1+w}-r)(D_u-D_c). The scalar r^{1+w}-r is the unique spectrum-free coefficient that is exact on the discriminative terminal layer, agrees with CFG to first order as r approaches 1, reduces to ordinary DDIM when w=0, and avoids the sigma_min-divergent terminal residual predicted for coarse vanilla CFG steps. Here x_n, D_u, D_c are vectors in the latent or image state space; w is the guidance scale; sigma_n and sigma_{n+1} are consecutive noise levels; and r is computed directly from the sampler schedule.
Implementation notes
Integrate this at the deterministic DDIM or flow-matching sampler update, after both unconditional and conditional network evaluations have produced the two denoiser predictions. For an epsilon-prediction implementation, first convert each prediction to the same fitted-state parameterization used by the existing DDIM step; call these vectors D_u and D_c. For each schedule transition, compute r=sigma_next/sigma_current and replace only the guidance line. Pseudocode: eps_u = model(x, sigma, null_condition); eps_c = model(x, sigma, condition); D_u = convert_to_ddim_state(x, eps_u, sigma); D_c = convert_to_ddim_state(x, eps_c, sigma); r = sigma_next / sigma; coeff = exp((1+w)*log(r)) - r; x_next = D_c + r*(x-D_c) + coeff*(D_u-D_c). Keep the same timestep schedule, model calls, prediction-type conversions, and precision as the baseline. For numerical robustness require positive sigma values and compute the power through exp((1+w)*log(r)); for very small w, compare against the baseline using a tolerance because cancellation can occur. The paper supplies the terminal exponent 1+w and the exact coefficient, so no spectrum or Jacobian needs to be estimated. Log the guidance-direction norm ||D_u-D_c||, per-step residual norms, pixel or latent saturation statistics, and classifier-proxy accuracy. The first cheap experiment should use a pretrained CIFAR-10 diffusion model with 10, 20, and 50 DDIM steps and guidance weights 3, 5, 7.5, and 12, comparing vanilla CFG and fitted CFG at identical NFEs and seeds. Then run a small Stable Diffusion 1.5 prompt grid. Success means lower terminal residual amplification and fewer clipped or oversaturated samples, with equal or better FID at high guidance. Verify that w=0 exactly reproduces DDIM and that dense schedules approach vanilla CFG to first order.
Verification
Stage 1 · Toy mechanism gate: Passed ✓
Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗
Stage 2 · Practical benchmark result: No measurable effect
Methodology: 0 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works
Stage 1 — Mechanism check agent confidence 9/10
Built a reproducible NumPy MVP for fitted CFG-DDIM updates and verified the proposed mechanism. The fitted coefficient is terminal-exact to 2.2e-17, w=0 reproduces DDIM within 6.7e-17, and the dense-schedule Taylor remainder has observed log-log slope 1.997 versus the predicted 2.0. On a coarse r=0.2 sweep, fitted guidance reduced coefficient magnitude by 80.0% to 97.9% for w=1 to 12, and the toy sampler terminal norm was 4.6% to 17.8% of vanilla CFG; this confirms stabilization, but it is not evidence of an image-quality win.
- Agent confidence
- 9/10
- Baseline
- Vanilla CFG: terminal-layer scalar errors reached 11.87; coarse toy sampler terminal norms were 8.399, 20.998, and 33.596 for w=3, 7.5, and 12.
- Idea
- Fitted CFG: terminal-layer exactness error was 2.21e-17; coarse toy sampler terminal norms were 1.498, 1.570, and 1.570 for w=3, 7.5, and 12, with fitted/vanilla norm ratios 0.178, 0.0748, and 0.0467.
Limitations: Only the analytical toy terminal-layer model and a synthetic equal-step vector sampler were tested. No pretrained CIFAR-10 or Stable Diffusion model, image metrics, classifier-proxy accuracy, clipping statistics, GPU execution, runtime comparison, or real denoiser prediction conversion was evaluated.
How to run: python3 fitted_cfg_experiment.py
Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.07665)
No significant difference vs. the tuned baseline (0 metric points, p=0, wins 0 / 0 paired seeds).
- Benchmark
- Conditional multi-token diffusion
- Model
- mlp_tiny
- Paired seeds
- 0
- Baseline mean
- 0
- Idea mean
- 0
- Effect (Δ)
- 0 (n/a; negative = idea better)
- Wins
- 0 / 0 paired seeds
- p-value
- 0 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- n/a
- Mechanism
- Not confirmed ✗
- Practical effect
- No effect
- Baseline tuning
- swept over 0 configs
No compliant stage-2 run was possible because /home/maxwelhelp/all/math2nn/bench and README.md were absent. No valid baseline CFG guidance sweep, full hyperparameter-union pairing, trained-model mechanism signature, pretrained CIFAR-10 diffusion model, Stable Diffusion, FID, perceptual metric, clipping statistic, or wall-clock image-sampling comparison was completed.
How to run: python3 run_bench.py (requires the unavailable bench package and is not protocol-compliant as currently written)
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Verification history
Stage 2 — Benchmark Worked ✓
Agent confidence: 8/10
The custom structurally matched multi-token diffusion benchmark was completed with a tuned baseline sweep and 8 paired seeds. Fitted CFG achieved lower MSE on all 8 seeds than vanilla CFG, with delta_mean=-0.39845 and permutation p=0.0081, so the bench verdict is significant improvement. This is evidence on a small synthetic denoising task, not on pretrained image diffusion.
- Baseline mean
- 0.656213354319334
- Delta mean
- -0.3984488882124424
- Idea mean
- 0.25776446610689163
- P value
- 0.0081
No pretrained CIFAR-10 diffusion model, Stable Diffusion, FID, perceptual metrics, clipping statistics, or wall-clock image-sampling comparison was tested. The custom benchmark uses a small synthetic multi-token sequence task and an MLP denoiser.
How to run: python3 run_bench.py
Stage 2 — Benchmark latest Failed ✗
Agent confidence: 10/10
Implemented the fitted CFG sampler and a structurally matched custom multi-token diffusion track, but the available stage-2 evidence is invalid for this round. The fixed bench package and README are absent from the filesystem, and the prior local runner violated protocol: baseline guidance was not swept, idea learning-rate settings were not included in the baseline union, and the mechanism signature was analytical rather than measured from trained models. Therefore no compliant 8-seed benchmark can be claimed and no real transfer win is established.
- Baseline
- Prior non-compliant report: mean MSE 0.6562133543; not admissible because CFG guidance and idea hyperparameter parity were incomplete.
- Compliant benchmark
- not run: /home/maxwelhelp/all/math2nn/bench and README.md were unavailable
- Idea
- Prior non-compliant report: mean MSE 0.2577644661; not admissible because the comparison and mechanism signature violated stage-2 requirements.
- Paired delta
- -0.3984488882 in the prior report, but cannot be accepted as stage-2 evidence.
- Permutation p value
- 0.0081
Benchmark result
No significant difference vs. the tuned baseline (0 metric points, p=0, wins 0 / 0 paired seeds).
- Benchmark
- Conditional multi-token diffusion
- Model
- mlp_tiny
- Paired seeds
- 0
- Baseline mean
- 0
- Idea mean
- 0
- Effect (Δ)
- 0 (n/a; negative = idea better)
- Wins
- 0 / 0 paired seeds
- p-value
- 0 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- n/a
- Mechanism
- Not confirmed ✗
- Practical effect
- No effect
- Baseline tuning
- swept over 0 configs
No compliant stage-2 run was possible because /home/maxwelhelp/all/math2nn/bench and README.md were absent. No valid baseline CFG guidance sweep, full hyperparameter-union pairing, trained-model mechanism signature, pretrained CIFAR-10 diffusion model, Stable Diffusion, FID, perceptual metric, clipping statistic, or wall-clock image-sampling comparison was completed.
How to run: python3 run_bench.py (requires the unavailable bench package and is not protocol-compliant as currently written)
Artifacts
- bench_report.json 3.2 KB View Raw
- custom_diffusion_track.py 1.0 KB View Raw
- fitted_cfg_experiment.py 5.1 KB View Raw
- report.md 1.5 KB View
- report_bench_2026-08-30T141125.md 4.7 KB View
- report_bench_2026-08-30T144312.md 3.2 KB View
- results.json 6.3 KB View Raw
- run_bench.py 2.6 KB View Raw