△ Mechanism confirmed, baseline not beaten 2026

Fisher-Geodesic Finite-Step Annealing

Implementation & benchmark of arXiv:2608.25207 — Finite relaxation protocols with minimal dissipation

Usefulness7/10

Difficulty5/10

Novelty7/10

Sampling Speedup Stability Sample efficiency Diffusion sampling Scheduler Training dynamics Statistical mechanics Information theory Geometry Stochastic processes Dynamical systems

Source paper: Finite relaxation protocols with minimal dissipation arXiv:2608.25207 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace uniformly spaced diffusion or energy annealing schedules by a finite sequence of quench-relax stages whose intermediary distributions are approximately equally spaced in Fisher-Rao distance. Each stage abruptly changes the model energy or noise level and then runs a short relaxation phase; the schedule concentrates stages where the distribution changes most sharply. This should reduce nonequilibrium mismatch at a fixed number of sampler evaluations and avoid large distributional jumps that destabilize Langevin or reverse-diffusion integration.

Formulas

$$\pi^{H}(x)=\frac{e^{-\beta H(x)}}{Z_H},\qquad Z_H=\sum_x e^{-\beta H(x)},\qquad F_H(\rho)-F_H(\pi^H)=\beta^{-1}D_{\mathrm{KL}}(\rho\|\pi^H).$$

$$ds^2=d\theta^{\mathsf T}I(\theta)d\theta,\qquad I(\theta)=\mathbb E_{x\sim p_\theta}\left[\nabla_\theta\log p_\theta(x)\nabla_\theta\log p_\theta(x)^{\mathsf T}\right].$$

$$\theta_k:\quad s(\theta_k)\approx \frac{k}{N}L,\qquad L=\int_0^1\sqrt{\dot\theta(u)^{\mathsf T}I(\theta(u))\dot\theta(u)}\,du.$$

$$\sum_{k=0}^{N-1}D_{\mathrm{KL}}(p_{\theta_k}\|p_{\theta_{k+1}})=\frac{1}{2}\sum_{k=0}^{N-1}(\Delta s_k)^2+O(N^{-2})\approx\frac{L^2}{2N}.$$

Mathematical statement

For Hamiltonian H(x), inverse temperature beta, and state x, the equilibrium distribution is pi^H(x)=exp(-beta H(x))/Z_H, where Z_H=sum_x exp(-beta H(x)). For an arbitrary distribution rho, the excess free energy satisfies F_H(rho)-F_H(pi^H)=beta^{-1}D_KL(rho||pi^H), so nonequilibrium error is measured by KL divergence. For a differentiable path p_theta(x), the Fisher-Rao line element is ds^2=dtheta^T I(theta)dtheta, where I(theta)=E_{x~p_theta}[grad_theta log p_theta(x) grad_theta log p_theta(x)^T] is the Fisher information matrix. The paper's finite-protocol result states that optimal intermediary equilibria converge to a Fisher-Rao geodesic as the number N of intermediary states increases. Thus choose theta_k so that the arc lengths Delta s_k are approximately equal, with Delta s_k approximately L/N and L=int_0^1 sqrt(theta'(u)^T I(theta(u))theta'(u))du. Locally, KL(p_{theta_k}||p_{theta_{k+1}})=1/2 Delta s_k^2+O(Delta s_k^3), so equal arc-length spacing gives cumulative leading-order mismatch approximately L^2/(2N), whereas a nonuniform parameter schedule wastes steps in low-curvature regions.

Implementation notes

1. Integration point: use this as a scheduler for a diffusion sampler, annealed Langevin sampler, or energy-based model. Let p_theta denote the model distribution along a one-dimensional annealing path theta in [0,1], where theta can be log-SNR, inverse temperature, noise scale, or an interpolation coefficient between a base Gaussian and the data distribution. Keep the neural score or energy network unchanged and replace only the usual linear or cosine schedule with an adaptive list of N schedule values.

2. Pseudocode: construct a dense pilot grid theta[0:M]. At every grid point estimate the scalar Fisher speed v_j=sqrt(E[(d/dtheta log p_theta(x))^2]) using minibatch samples and automatic differentiation through the energy, score, or denoising parameterization. Form cumulative lengths s_j=s_{j-1}+v_j(theta_j-theta_{j-1}), then select N+1 schedule points by inverse interpolation at s=k*s_M/N. During sampling, quench to theta_k and apply r_k relaxation updates, such as x<-x+epsilon*s_theta_k(x)+sqrt(2epsilon)xi for Langevin. Initially use constant r_k; later test r_k proportional to the local estimated KL or Fisher speed.

3. Computed from the paper are the Fisher metric, equal-arc-length rule, and inverse-N leading-order dissipation scaling. Estimated empirically are Fisher speeds, one-step KL proxies, and relaxation errors. A scalar or diagonal Fisher estimate is sufficient for the first implementation.

4. First cheap experiment: compare linear, cosine, and Fisher-geodesic schedules using a small pretrained CIFAR-10 diffusion score model at N=10, 20, 40 function evaluations. Measure FID, reverse-process KL proxies, and cumulative one-step KL. The quantitative prediction is cumulative KL approximately C/N for the geodesic schedule, while each local KL should be nearly equal across stages. The measured local-KL coefficient of variation should be lower than for parameter-uniform schedules, and the fitted exponent in cumulative KL versus N should approach -1 over the discretization-dominated regime.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

The Fisher identity matched Monte Carlo within 0.066%. Equal-Fisher spacing reduced local-KL coefficient of variation to approximately zero, versus 0.406 to 4.531 for linear spacing, and the widest-range cumulative KL approached the predicted L^2/(2N) scaling.

Agent confidence: 9/10

Baseline: Linear variance schedule: widest-range N=32 cumulative KL was 3.90 times the geodesic result; fitted cumulative-KL exponent was -0.478 over N=16..128.
Idea: Equal-Fisher-arc geometric-variance schedule: local-KL CV below 4e-15; fitted exponent -0.950 over N=16..128; N*KL=8.966 at N=2048 versus predicted 8.974.

Limitations: Only a one-dimensional exact Gaussian path was tested. No neural diffusion model, Langevin relaxation, FID, runtime, noisy Fisher estimation, or multidimensional learned-energy experiment was tested.

How to run: python3 fisher_annealing_toy.py && python3 extra_check.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.25207) Sep 2, 2026

Significantly WORSE than the tuned baseline (+3.4%, p=0.0307, wins 1 / 8 paired seeds).

Benchmark: Fisher_multitoken_denoising
Model: custom_denoiser
Paired seeds: 8
Baseline mean: 0.3757
Idea mean: 0.3885
Effect (Δ): 0.0128 (+3.4%; negative = idea better)
Wins: 1 / 8 paired seeds
p-value: 0.0307 (permutation test, 20 000 shuffles)
Smallest detectable effect: n/a
Mechanism: Confirmed ✓
Practical effect: Hurts
Baseline tuning: swept over 0 configs

Limitations:

This was a small custom synthetic multitoken denoising task with 400 training examples, 160 test examples, a simple denoiser, 18 epochs, and one schedule length. CIFAR-10 diffusion, pretrained score models, FID, Langevin relaxation variants, larger models, and noisy Fisher estimation were not tested.

How to run: python3 run_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 2, 2026 latest Failed ✗

Agent confidence: 9/10

Built and ran a custom correlated multi-token denoising benchmark with matched neural denoisers. Fisher-geodesic spacing reduced trained-model stage-error CV from 0.61044 to 0.47896 and confirmed the analytic equal-arc prediction, but test MSE was significantly worse: 0.38846 versus 0.37566, paired delta +0.01280 and permutation p=0.03075. Therefore the mechanism transferred, but the promised task-level improvement did not.

Baseline: Test MSE mean 0.3756636306643486 over 8 seeds; best sweep config lr=0.003, smax=1.0.
Idea: Test MSE mean 0.38846133276820183 over 8 seeds; paired delta +0.012797702103853226; permutation p=0.03075; 1/8 paired wins.
Mechanism signature: map[confirmed:true observed_parameter_path_linear_kl_cv:0.6206565234045026 predicted_equal_arc_local_kl_cv:1.7439286709141664e-06 trained_model_fisher_stage_error_cv:0.4789595501048241 trained_model_linear_stage_error_cv:0.610444495224178]

Benchmark result