Ball-Coordinate MCMC for Heavy-Tailed Bayesian Networks
Implementation & benchmark of arXiv:2608.04284 — Diffeomorphic Markov Chain Monte Carlo: fast mixing for heavy-tailed distributions
Source paper: Diffeomorphic Markov Chain Monte Carlo: fast mixing for heavy-tailed distributions arXiv:2608.04284 ⓘ · analyzed Aug 31, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Reparameterize all Bayesian neural-network weights by a bounded latent vector in the unit ball and use a simple ball-constrained MCMC kernel instead of unconstrained HMC or random-walk sampling. A radial diffeomorphism expands points near the ball boundary into arbitrarily large weights, preserving heavy-tailed posterior mass while preventing the sampler from numerically wandering through an unbounded parameter space.
Formulas
Mathematical statement
The paper defines the radial map ${\rm RC}_{\beta}:B(1)\to\mathbb{R}^{d}$ by ${\rm RC}_{\beta}(y)=\phi_{\beta}(|y|)y$, where $B(1)=\{y\in\mathbb{R}^{d}:|y|<1\}$, $d$ is the number of Bayesian parameters, and $\beta>0$ controls radial tail growth. Its Jacobian determinant is $J_{{\rm RC}_{\beta}}(y)=\phi_{\beta}(|y|)^{d-1}[\phi_{\beta}(|y|)+\phi_{\beta}'(|y|)|y|]$. The required boundary conditions are $(1-t)\phi_{\beta}(t)^{\beta}\to c$ and $(1-t)\phi_{\beta}'(t)/\phi_{\beta}(t)\to c'$ as $t\uparrow1$, for positive constants $c,c'$. Use the explicit choice $\phi_{\beta}(t)=(1-t)^{-1/\beta}$, for which $c=1$ and $c'=1/\beta$. If the desired weight posterior density is $\pi(w)$ and $w={\rm RC}_{\beta}(y)$, then the transformed density on the ball is $\tilde\pi(y)\propto\pi({\rm RC}_{\beta}(y))J_{{\rm RC}_{\beta}}(y)$. The Metropolis-Hastings target must therefore contain both the original log posterior and the log Jacobian.
Implementation notes
Integrate this at the Bayesian-parameter sampling layer, not inside the forward network. Flatten all stochastic weights and biases into one vector $w\in\mathbb{R}^{d}$, maintain a latent vector $y\in B(1)$, and compute network predictions using $w={\rm RC}_{\beta}(y)$. For each step, calculate $r=\|y\|_2$, set $\phi=(1-r)^{-1/\beta}$, and return $w=\phi y$; use $w=0$ when $r$ is numerically zero. Propose $z=y+\sigma\epsilon$ with $\epsilon\sim\mathcal{N}(0,I_d)$. Reject immediately if $\|z\|_2\ge1$. Otherwise compute $w'={\rm RC}_{\beta}(z)$, evaluate the network likelihood and weight-prior density, and add the log-Jacobian to form $\log\tilde\pi(z)=\log p(D\mid w')+\log p(w')+\log J(z)$. Accept with the displayed Metropolis ratio; for symmetric Gaussian proposals the proposal terms cancel. Compute the Jacobian stably in log space using $\log J=(d-1)\log\phi+\log(\phi+\phi' r)$, where $\phi'=\phi/[\beta(1-r)]$. Tune $\sigma$ and $\beta$ empirically; optionally replace isotropic noise with a diagonal preconditioner estimated from warmup samples. The first experiment should use a two-layer MLP on a small UCI regression dataset or MNIST with Student-t weight priors. Compare transformed random-walk MH against unconstrained random-walk MH and NUTS at equal likelihood evaluations. Measure predictive RMSE or accuracy, uncertainty calibration, effective sample size per evaluation, integrated autocorrelation time, rejection rate, and numerical divergences. Success means higher effective samples and fewer unstable trajectories, especially under artificially heavy-tailed priors; boundary concentration near $\|y\|=1$ is a falsifiable failure mode.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.