# State-Dependent Temperature Langevin

- ID: 2845
- Canonical URL: https://synthcore.org/idea/2845/state-dependent-temperature-langevin
- API JSON: https://synthcore.org/api/idea/2845.json
- API Markdown: https://synthcore.org/api/idea/2845.md
- Verification status: unverified
- Source: [arXiv:2608.29657](https://arxiv.org/abs/2608.29657)
- Category: sampling
- Solves: speedup, stability, sample-efficiency
- ML areas: diffusion-sampling, vae, world-model
- Math tags: probability, stochastic-processes, functional-analysis, geometry
- Ratings: usefulness 7/10; difficulty 4/10; novelty 6/10

## Idea description

Replace isotropic Langevin noise in latent or energy-based neural sampling with a smooth position-dependent temperature \(\sigma(x)\geq 1\). Use the divergence correction associated with the diffusion matrix so that increasing exploration in the tails does not change the desired target distribution. This should reduce metastability and improve effective samples per gradient evaluation on heavy-tailed latent posteriors.

## Mathematical statement

The paper defines a target position law \(\mu(dx)=e^{-U(x)}dx/Z_U\), where \(x\in\mathbb R^d\) is the sampled state, \(U:\mathbb R^d\to\mathbb R\) is an energy, and \(Z_U=\int e^{-U(x)}dx\) is an unknown normalizer. It introduces the conditional velocity law \(\kappa_x(dv)=\frac{\exp(-|v|^2/(2\sigma^2(x)))}{(2\pi\sigma^2(x))^{d/2}}dv\), where \(v\in\mathbb R^d\) and \(\sigma:\mathbb R^d\to[1,\infty)\) is smooth and Lipschitz. The joint invariant density is therefore \(\mu_\sigma(dx,dv)=\mu(dx)\kappa_x(dv)\): the temperature changes the auxiliary dynamics but not the desired marginal over \(x\). We transplant the corresponding state-dependent diffusion matrix \(a(x)=\sigma^2(x)I_d\). For the Itô diffusion \(dX_t=b(X_t)dt+\sqrt{2a(X_t)}dW_t\), choosing \(b=\nabla\!\cdot a-a\nabla U\) makes \(\pi(x)\propto e^{-U(x)}\) invariant because the stationary probability current \(b\pi-\nabla\!\cdot(a\pi)\) vanishes. Since \(a=\sigma^2I_d\), \(\nabla\!\cdot a=\nabla\sigma^2\), giving \(b(x)=\nabla\sigma^2(x)-\sigma^2(x)\nabla U(x)\). Here \(W_t\) is standard \(d\)-dimensional Brownian motion. The paper's smooth-Lipschitz requirement on \(\sigma\) supplies bounded, numerically stable coefficients; its weighted-Poincare and hypocoercive framing motivates evaluating convergence on heavy-tailed targets.

## Key formulas

- $$\mu(dx)=\exp(-U(x))\frac{dx}{Z_U},\qquad Z_U=\int_{\mathbb R^d}e^{-U(x)}dx.$$
- $$\kappa_x(dv)=\frac{\exp\left(-|v|^2/(2\sigma^2(x))\right)}{(2\pi\sigma^2(x))^{d/2}}\,dv,\qquad \mu_\sigma(dx,dv)=\mu(dx)\kappa_x(dv).$$
- $$dX_t=\left[\nabla\!\cdot a(X_t)-a(X_t)\nabla U(X_t)\right]dt+\sqrt{2a(X_t)}\,dW_t,\qquad a(x)=\sigma^2(x)I_d.$$
- $$dX_t=\left[\nabla\sigma^2(X_t)-\sigma^2(X_t)\nabla U(X_t)\right]dt+\sqrt{2}\,\sigma(X_t)dW_t.$$

## Implementation notes

(1) Integration point: modify the predictor step of a Langevin-based latent sampler, diffusion corrector, or posterior sampler. The state tensor is \(x\in\mathbb R^{B\times d}\); obtain \(U(x)\) and \(g=\nabla_xU(x)\) by automatic differentiation. Do not apply this to the neural-network parameter optimizer initially; use it for latent states or generated samples. Choose a smooth temperature such as \(\sigma(x)=1+\alpha\log(1+\|x\|_2/\sqrt d)\), with \(\alpha\geq0\), or a clipped affine function of a frozen score norm. This is smooth, at least one, and has bounded derivative. (2) Pseudocode: `for k in 1..K: x.requires_grad=True; U=energy(x); g=grad(U,x); s=1+alpha*log1p(norm(x)/sqrt(d)); sigma2=s*s; grad_sigma2=grad(sigma2.sum(),x); z=normal_like(x); x=x+eps*(grad_sigma2-sigma2*g)+sqrt(2*eps)*s*z; x=stop_gradient(x)`. The term `grad_sigma2` is the divergence correction \(\nabla\!\cdot a\), not optional if exact target preservation matters. (3) Computed from the mathematics: the drift \(\nabla\sigma^2-\sigma^2\nabla U\) and noise scale \(\sqrt2\sigma\). Estimated empirically: integrated autocorrelation time, effective sample size per gradient call, bias of sample moments, and acceptance if an optional Metropolis correction is added. Use a small step size first; for learned energies with discretization bias, add a Metropolis-adjusted Langevin correction or compare against a long-run baseline. (4) First experiment: sample a 20- or 50-dimensional Student-t target with known degrees of freedom and compare ordinary ULA/MALA against this sampler at matched energy-gradient evaluations. Then test a VAE latent posterior or a 2D/8D neural energy model. Plot ESS per gradient evaluation, tail-quantile error, and Wasserstein distance versus wall-clock time. Success means at least 2x larger ESS at equal gradient calls, without systematic bias in known Student-t moments or unstable trajectories.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.