Idea description

Replace uniformly spaced diffusion timesteps with a grid whose intervals contribute equal area under the local information-loss curve. The sampler then takes smaller steps in noise regions where the denoiser contributes most to likelihood and larger steps in low-information regions.

Formulas

$$H(X)=H(X\mid Y(1))-H(X\mid Y(0))=\sum_{j=0}^{m-1}\left[H(X\mid Y(\tau_{j+1}))-H(X\mid Y(\tau_j))\right]$$

$$H(X)=\int_0^1 \frac{\partial}{\partial\tau}H(X\mid Y(\tau))\,d\tau$$

$$A(\tau)=\int_0^\tau |g(u)|\,du,\qquad g(\tau)=\frac{\partial}{\partial\tau}H(X\mid Y(\tau)),\qquad \tau_k=A^{-1}\left(\frac{k}{K}A(1)\right)$$

$$\widehat g(\tau_j)\approx\frac{\widehat H(X\mid Y(\tau_{j+1}))-\widehat H(X\mid Y(\tau_j))}{\tau_{j+1}-\tau_j}$$

Mathematical statement

The paper uses the telescoping identity $H(X)=H(X\mid Y(1))-H(X\mid Y(0))=\sum_{j=0}^{m-1}[H(X\mid Y(\tau_{j+1}))-H(X\mid Y(\tau_j))]$ and, under absolute continuity of $\tau\mapsto H(X\mid Y(\tau))$, obtains $H(X)=\int_0^1 \partial_\tau H(X\mid Y(\tau))\,d\tau$. Here $X$ is a clean sample, $Y(\tau)$ is its output under a memoryless channel at noise level $\tau$, and $g(\tau)=\partial_\tau H(X\mid Y(\tau))$ is the local information-dissipation density. Define $A(\tau)=\int_0^\tau |g(u)|\,du$ and choose $K$ grid points by $\tau_k=A^{-1}(kA(1)/K)$. Thus each interval has approximately equal information area. The curve $g$ is estimated empirically from neighboring conditional-entropy or posterior-loss evaluations.

Implementation notes

Integrate this at the inference scheduler. First train a standard diffusion model with its existing noise parameterization. On a held-out calibration set, construct a dense monotone grid of native noise values $\tau_0<\cdots<\tau_M$. For every grid point, corrupt the same examples and evaluate the model's posterior loss, producing $\widehat H_j=-N^{-1}\sum_n\log q_\theta(x_n\mid y_n(\tau_j),\tau_j)$. For Gaussian diffusion, use the corresponding denoising half-MSE in an SNR coordinate instead. Estimate the local curve using the finite-difference formula above, smooth $|\widehat g_j|$ with a short moving average, and compute cumulative trapezoidal areas $A_j$. For a requested budget of $K$ reverse evaluations, interpolate the inverse cumulative function at $kA_M/K$ to obtain the new grid. Map those values back to the model's native timestep or SNR input and use the existing reverse update between adjacent points. The conservation law is mathematical; the curve, smoothing, and inverse interpolation are empirical. Begin with a 50M-parameter CIFAR-10 DDPM or a small discrete masked diffusion model. Compare linear, cosine, and equal-information grids at 10, 20, and 50 function evaluations. Measure FID, validation NLL, and wall-clock time. The expected signal is lower FID or NLL at fixed function evaluations, ideally allowing a 2x reduction in steps at matched quality.

Verification

Beats tuned baseline

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Significant win ✓

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a reusable equal-information scheduler in equal_information_grid.py and verified it against an analytic conditional-entropy curve. The three predictions were confirmed: equal-area CV was approximately 1e-15, geometric spacing matched to at most 6.7e-16, and the uniform/equal-information midpoint-error ratio grew from 2.95 at eps=0.1 to 9.28 at eps=0.001. At K=10 and eps=0.01, the toy entropy-integral error fell from 0.1923 for a uniform grid to 0.0248 for the proposed grid, but this is not evidence of improved image-sampling quality.

Agent confidence: 8/10

Baseline: Uniform grid, K=10, eps=0.01: interval-area CV=1.2393; midpoint entropy-integral error=0.192338
Idea: Equal-information grid, K=10, eps=0.01: interval-area CV=6.8e-16; midpoint entropy-integral error=0.024756; 7.77x lower toy integration error

Limitations: No trained diffusion model, reverse sampler, CIFAR-10 data, FID, validation NLL, wall-clock comparison, cosine schedule, or DDPM native timestep mapping was tested. The noisy calibration check showed degradation at entropy noise std=0.01 (area CV=0.444), so empirical estimation stability remains an important practical limitation.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.10067) Sep 2, 2026

Beats the tuned baseline by 98.5% (p=0.0081, wins 8 / 8 paired seeds; metric: lower is better).

Benchmark: Conditional multi-token diffusion
Model: diffusion_mlp_tiny
Paired seeds: 8
Baseline mean: 132.7725
Idea mean: 2.007
Effect (Δ): -130.7655 (−98.5%; negative = idea better)
Wins: 8 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±39.8%
Mechanism: Confirmed ✓
Practical effect: Helps
Baseline tuning: swept over 3 configs

Limitations:

No CIFAR-10 DDPM, FID, validation NLL, wall-clock measurement, cosine schedule, or production DDIM/DPM-Solver comparison was tested. The registered task is a small synthetic correlated multi-token diffusion benchmark, and the extreme baseline error indicates possible numerical sensitivity of the uniform reverse Euler implementation; broader sampler baselines are needed before claiming general diffusion superiority.

How to run: python3 equal_information_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 31, 2026 latest Worked ✓

Agent confidence: 6/10

The registered-bench evaluation reports a significant win for equal-information diffusion sampling: mean test MSE 2.00695 versus 132.77249 for the tuned uniform-grid baseline, with all 8 paired wins and permutation p=0.0081. The trained-model calibration signature supports the proposed mechanism, reducing area CV from 0.7855 to 0.0646. The magnitude is unusually large and likely reflects instability of the uniform reverse Euler baseline near the terminal noise level, so this is evidence on the registered synthetic track rather than production DDPM evidence.

Baseline: conditional_multitoken_diffusion, tuned uniform grid, lr=0.009, epochs=6, K=12: mean test MSE 132.77248764038086, std 58.82544740704864
Delta mean: -130.76553366333246
Idea: conditional_multitoken_diffusion, equal-information grid, lr=0.001, epochs=6, K=12: mean test MSE 2.006953977048397, std 1.400844895081476
Mechanism signature: map[calibration_area_cv:0.06464579480072169 confirmed:true uniform_reference_cv:0.7855128413469473]
P value: 0.0081
Paired wins: 8/8

Benchmark result