Accelerated randomized Hamiltonian posterior sampler
Implementation & benchmark of arXiv:2607.12902 — Accelerated Mixing Time of Randomized Hamiltonian Monte Carlo
Source paper: Accelerated Mixing Time of Randomized Hamiltonian Monte Carlo arXiv:2607.12902 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace Langevin or random-walk sampling for a strongly log-concave neural subproblem with randomized Hamiltonian trajectories. Each iteration draws a fresh Gaussian velocity, integrates position and velocity for a random triangular or exponential duration, and discards the terminal velocity before the next refresh. The target is a regularized posterior over a convex neural-network head, where the paper's accelerated dependence on the strong-convexity parameter is applicable.
Formulas
Mathematical statement
The target distribution is nu^X(dx) proportional to exp(-f(x)) dx, where x in R^d is the sampled parameter vector and f is its negative log posterior. The phase-space Hamiltonian is H(x,y)=f(x)+1/2||y||^2, with position x, velocity y, and Gaussian kinetic energy. Hamiltonian flow follows dX_t/dt=Y_t and dY_t/dt=-gradient f(X_t), so the sampler performs a second-order trajectory instead of a first-order random walk. Between trajectories, set Y_0 distributed as N(0,I_d) independently; this refresh preserves the Gaussian velocity marginal and removes accumulated directional bias. For a triangular duration with mean tau=Theta(alpha^{-1/2}), use density p_tau(t)=t/tau^2 for 0<=t<=tau and p_tau(t)=(2tau-t)/tau^2 for tau<t<=2tau. An exponential duration has density p_tau(t)=tau^{-1} exp(-t/tau). Here alpha>0 is the strong-convexity or Talagrand constant. The relevant assumption is the alpha-Talagrand inequality KL(mu||nu)<=W_2(mu,nu)^2/(2 alpha), where KL is relative entropy and W_2 is the 2-Wasserstein distance. Under log-concavity and this inequality, the paper gives total integration time O(alpha^{-1/2} log(1/epsilon)) to reach KL error epsilon. In code, exact flow is replaced by a symplectic integrator, so a Metropolis correction or sufficiently small integration error is needed.
Implementation notes
Integrate this at the sampling component for a Bayesian neural-network head, not at ordinary deterministic SGD. First train a feature extractor and freeze it. Let z_i be its features and let w be the final linear or small MLP-head parameters. Use a Gaussian prior and a convex likelihood such as binary logistic regression or squared-error regression, giving f(w)=-log p(D|w)+(lambda/2)||w||^2. For logistic regression, estimate a conservative strong-convexity lower bound alpha>=lambda and set tau=c/sqrt(alpha), with c swept over 0.25, 0.5, 1, and 2. At every sampler iteration: (1) draw y from N(0,I); (2) draw T from the triangular density above, or from an exponential distribution with mean tau; (3) choose leapfrog step size h and integer L=ceil(T/h); (4) perform L leapfrog updates, using y <- y-(h/2) gradient f(w), w <- w+h y, and y <- y-(h/2) gradient f(w); (5) accept with probability min(1, exp(-H(w',y')+H(w,y))), or monitor energy error if Metropolis correction is too expensive; (6) store w' and discard y'. The paper supplies the alpha^{-1/2} trajectory-scale rule and the mixing target; estimate alpha, gradient noise, acceptance rate, and effective sample size empirically. Start with MNIST or CIFAR-10 frozen ResNet features and a Bayesian logistic head. Compare against unadjusted Langevin, preconditioned Langevin, and fixed-length HMC at equal gradient evaluations. Success means higher effective samples per gradient evaluation, faster decay of predictive posterior error, and stable acceptance above 0.7; test whether the advantage grows as alpha decreases.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.