△ Mechanism confirmed, baseline not beaten 2026

State-Dependent Temperature Langevin

Implementation & benchmark of arXiv:2608.29657 — Hypocoercivity of Tempered Bouncy Particle Samplers for Heavy-Tailed Targets

Usefulness7/10

Difficulty4/10

Novelty6/10

Sampling Speedup Stability Sample efficiency Diffusion sampling Vae World model Probability Stochastic processes Functional analysis Geometry

Source paper: Hypocoercivity of Tempered Bouncy Particle Samplers for Heavy-Tailed Targets arXiv:2608.29657 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace isotropic Langevin noise in latent or energy-based neural sampling with a smooth position-dependent temperature $\sigma(x)\geq 1$. Use the divergence correction associated with the diffusion matrix so that increasing exploration in the tails does not change the desired target distribution. This should reduce metastability and improve effective samples per gradient evaluation on heavy-tailed latent posteriors.

Formulas

$$\mu(dx)=\exp(-U(x))\frac{dx}{Z_U},\qquad Z_U=\int_{\mathbb R^d}e^{-U(x)}dx.$$

$$\kappa_x(dv)=\frac{\exp\left(-|v|^2/(2\sigma^2(x))\right)}{(2\pi\sigma^2(x))^{d/2}}\,dv,\qquad \mu_\sigma(dx,dv)=\mu(dx)\kappa_x(dv).$$

$$dX_t=\left[\nabla\!\cdot a(X_t)-a(X_t)\nabla U(X_t)\right]dt+\sqrt{2a(X_t)}\,dW_t,\qquad a(x)=\sigma^2(x)I_d.$$

$$dX_t=\left[\nabla\sigma^2(X_t)-\sigma^2(X_t)\nabla U(X_t)\right]dt+\sqrt{2}\,\sigma(X_t)dW_t.$$

Mathematical statement

The paper defines a target position law $\mu(dx)=e^{-U(x)}dx/Z_U$, where $x\in\mathbb R^d$ is the sampled state, $U:\mathbb R^d\to\mathbb R$ is an energy, and $Z_U=\int e^{-U(x)}dx$ is an unknown normalizer. It introduces the conditional velocity law $\kappa_x(dv)=\frac{\exp(-|v|^2/(2\sigma^2(x)))}{(2\pi\sigma^2(x))^{d/2}}dv$, where $v\in\mathbb R^d$ and $\sigma:\mathbb R^d\to[1,\infty)$ is smooth and Lipschitz. The joint invariant density is therefore $\mu_\sigma(dx,dv)=\mu(dx)\kappa_x(dv)$: the temperature changes the auxiliary dynamics but not the desired marginal over $x$. We transplant the corresponding state-dependent diffusion matrix $a(x)=\sigma^2(x)I_d$. For the Itô diffusion $dX_t=b(X_t)dt+\sqrt{2a(X_t)}dW_t$, choosing $b=\nabla\!\cdot a-a\nabla U$ makes $\pi(x)\propto e^{-U(x)}$ invariant because the stationary probability current $b\pi-\nabla\!\cdot(a\pi)$ vanishes. Since $a=\sigma^2I_d$, $\nabla\!\cdot a=\nabla\sigma^2$, giving $b(x)=\nabla\sigma^2(x)-\sigma^2(x)\nabla U(x)$. Here $W_t$ is standard $d$-dimensional Brownian motion. The paper's smooth-Lipschitz requirement on $\sigma$ supplies bounded, numerically stable coefficients; its weighted-Poincare and hypocoercive framing motivates evaluating convergence on heavy-tailed targets.

Implementation notes

(1) Integration point: modify the predictor step of a Langevin-based latent sampler, diffusion corrector, or posterior sampler. The state tensor is $x\in\mathbb R^{B\times d}$; obtain $U(x)$ and $g=\nabla_xU(x)$ by automatic differentiation. Do not apply this to the neural-network parameter optimizer initially; use it for latent states or generated samples. Choose a smooth temperature such as $\sigma(x)=1+\alpha\log(1+\|x\|_2/\sqrt d)$, with $\alpha\geq0$, or a clipped affine function of a frozen score norm. This is smooth, at least one, and has bounded derivative. (2) Pseudocode: for k in 1..K: x.requires_grad=True; U=energy(x); g=grad(U,x); s=1+alpha*log1p(norm(x)/sqrt(d)); sigma2=s*s; grad_sigma2=grad(sigma2.sum(),x); z=normal_like(x); x=x+eps*(grad_sigma2-sigma2*g)+sqrt(2*eps)*s*z; x=stop_gradient(x). The term grad_sigma2 is the divergence correction $\nabla\!\cdot a$, not optional if exact target preservation matters. (3) Computed from the mathematics: the drift $\nabla\sigma^2-\sigma^2\nabla U$ and noise scale $\sqrt2\sigma$. Estimated empirically: integrated autocorrelation time, effective sample size per gradient call, bias of sample moments, and acceptance if an optional Metropolis correction is added. Use a small step size first; for learned energies with discretization bias, add a Metropolis-adjusted Langevin correction or compare against a long-run baseline. (4) First experiment: sample a 20- or 50-dimensional Student-t target with known degrees of freedom and compare ordinary ULA/MALA against this sampler at matched energy-gradient evaluations. Then test a VAE latent posterior or a 2D/8D neural energy model. Plot ESS per gradient evaluation, tail-quantile error, and Wasserstein distance versus wall-clock time. Success means at least 2x larger ESS at equal gradient calls, without systematic bias in known Student-t moments or unstable trajectories.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a state-dependent-temperature Langevin sampler for a Student-t target, including the divergence correction ∇σ² and a matched uncorrected baseline. The mechanism checks confirmed the predictions: stationary-current residual was exactly 0 across α, σ(1) had predicted slope 0.69314718 with 5e-9 maximum error, and the quadratic correction coefficient at x=2 was 0.73240819 observed versus 0.73240819 predicted, with finite-difference error 1.5e-10. The corrected sampler preserved the target variance across α (1.641–1.673 versus the exact 1.667), while the uncorrected sampler became increasingly biased (variance 1.078, 0.817, and 0.583 for α=0.5,1,2); however, no reliable ESS-per-gradient improvement was demonstrated.

Agent confidence: 8/10

Baseline: Uncorrected state-dependent-noise Langevin: second moment at α=0.5/1/2 = 1.078/0.817/0.583; ESS per gradient = 0.168/0.152/0.168.
Idea: Divergence-corrected sampler: second moment at α=0.5/1/2 = 1.673/1.641/1.656 versus exact 1.667; ESS per gradient = 0.168/0.150/0.168. Mechanism predictions: max current residual 0, sigma slope error 5e-9, quadratic-coefficient error 9.7e-10.

Limitations: Only a one-dimensional Student-t target and Euler-Maruyama discretization were tested; no MALA correction, high-dimensional target, neural energy, latent posterior, wall-clock benchmark, or robust long-chain autocorrelation analysis was included. The simple batch-means ESS estimate saturates at the independent-sample ceiling for several settings, so it should not be interpreted as evidence of a sampling-speed win.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.29657) Sep 2, 2026

Significantly WORSE than the tuned baseline (+17140.3%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0046
Idea mean: 0.7926
Effect (Δ): 0.788 (+17140.3%; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±9246.8%
Mechanism: Confirmed ✓
Practical effect: Hurts
Baseline tuning: swept over 6 configs

Limitations:

The built-in bench lacks a latent posterior or energy-based sampling task, so ESS, metastability, Wasserstein distance, MALA acceptance, and latent sampling quality were not tested. The mechanism signature tests the coefficient scaling in the implemented update but does not validate sampling invariance on a neural latent posterior.

How to run: python3 bench_experiment.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 2, 2026 latest Failed ✗

Agent confidence: 10/10

The registered dynamics track was rerun with 8 paired seeds, tuned baseline hyperparameters, matched rnn_small architecture, and idea settings using learning rates included in the baseline sweep. Baseline mean test MSE was 0.00459717 versus 0.79256819 for the best idea setting; paired delta was +0.78797102 with permutation p=0.0081, so the idea was significantly worse. The mechanism signature was included for transparency, but this parameter-space transfer does not establish the original latent-sampling claim.

Baseline: mean test MSE=0.0045971721; best config lr=0.002, weight_decay=0.0001
Idea: mean test MSE=0.7925681882; best config alpha=0.05, lr=0.0005, weight_decay=0.0001; paired delta=+0.7879710161; p=0.0081

Benchmark result