# Ball-Coordinate MCMC for Heavy-Tailed Bayesian Networks

- ID: 1783
- Canonical URL: https://synthcore.org/idea/1783/ball-coordinate-mcmc-for-heavy-tailed-bayesian-networks
- API JSON: https://synthcore.org/api/idea/1783.json
- API Markdown: https://synthcore.org/api/idea/1783.md
- Verification status: unverified
- Source: [arXiv:2608.04284](https://arxiv.org/abs/2608.04284)
- Category: sampling
- Solves: stability, sample-efficiency, scalability
- ML areas: sampling, optimizer, mlp
- Math tags: differential-geometry, probability, measure-theory, numerical-analysis
- Ratings: usefulness 6/10; difficulty 5/10; novelty 7/10

## Idea description

Reparameterize all Bayesian neural-network weights by a bounded latent vector in the unit ball and use a simple ball-constrained MCMC kernel instead of unconstrained HMC or random-walk sampling. A radial diffeomorphism expands points near the ball boundary into arbitrarily large weights, preserving heavy-tailed posterior mass while preventing the sampler from numerically wandering through an unbounded parameter space.

## Mathematical statement

The paper defines the radial map ${\rm RC}_{\beta}:B(1)\to\mathbb{R}^{d}$ by ${\rm RC}_{\beta}(y)=\phi_{\beta}(|y|)y$, where $B(1)=\{y\in\mathbb{R}^{d}:|y|<1\}$, $d$ is the number of Bayesian parameters, and $\beta>0$ controls radial tail growth. Its Jacobian determinant is $J_{{\rm RC}_{\beta}}(y)=\phi_{\beta}(|y|)^{d-1}[\phi_{\beta}(|y|)+\phi_{\beta}'(|y|)|y|]$. The required boundary conditions are $(1-t)\phi_{\beta}(t)^{\beta}\to c$ and $(1-t)\phi_{\beta}'(t)/\phi_{\beta}(t)\to c'$ as $t\uparrow1$, for positive constants $c,c'$. Use the explicit choice $\phi_{\beta}(t)=(1-t)^{-1/\beta}$, for which $c=1$ and $c'=1/\beta$. If the desired weight posterior density is $\pi(w)$ and $w={\rm RC}_{\beta}(y)$, then the transformed density on the ball is $\tilde\pi(y)\propto\pi({\rm RC}_{\beta}(y))J_{{\rm RC}_{\beta}}(y)$. The Metropolis-Hastings target must therefore contain both the original log posterior and the log Jacobian.

## Key formulas

- $$\phi_{\beta}(t)=(1-t)^{-1/\beta},\qquad (1-t)\phi_{\beta}(t)^{\beta}=1,\qquad (1-t)\frac{\phi_{\beta}'(t)}{\phi_{\beta}(t)}=\frac{1}{\beta}.$$
- $${\rm RC}_{\beta}(y)=\phi_{\beta}(|y|)y,\qquad J_{{\rm RC}_{\beta}}(y)=\phi_{\beta}(|y|)^{d-1}\left(\phi_{\beta}(|y|)+\phi_{\beta}'(|y|)|y|\right).$$
- $$\tilde\pi(y)=\pi({\rm RC}_{\beta}(y))J_{{\rm RC}_{\beta}}(y),\qquad y\in B(1).$$
- $$\alpha(y,y')=\min\left\{1,\frac{\tilde\pi(y')q(y\mid y')}{\tilde\pi(y)q(y'\mid y)}\right\},\qquad y'\sim q(\cdot\mid y).$$

## Implementation notes

Integrate this at the Bayesian-parameter sampling layer, not inside the forward network. Flatten all stochastic weights and biases into one vector $w\in\mathbb{R}^{d}$, maintain a latent vector $y\in B(1)$, and compute network predictions using $w={\rm RC}_{\beta}(y)$. For each step, calculate $r=\|y\|_2$, set $\phi=(1-r)^{-1/\beta}$, and return $w=\phi y$; use $w=0$ when $r$ is numerically zero. Propose $z=y+\sigma\epsilon$ with $\epsilon\sim\mathcal{N}(0,I_d)$. Reject immediately if $\|z\|_2\ge1$. Otherwise compute $w'={\rm RC}_{\beta}(z)$, evaluate the network likelihood and weight-prior density, and add the log-Jacobian to form $\log\tilde\pi(z)=\log p(D\mid w')+\log p(w')+\log J(z)$. Accept with the displayed Metropolis ratio; for symmetric Gaussian proposals the proposal terms cancel. Compute the Jacobian stably in log space using $\log J=(d-1)\log\phi+\log(\phi+\phi' r)$, where $\phi'=\phi/[\beta(1-r)]$. Tune $\sigma$ and $\beta$ empirically; optionally replace isotropic noise with a diagonal preconditioner estimated from warmup samples. The first experiment should use a two-layer MLP on a small UCI regression dataset or MNIST with Student-t weight priors. Compare transformed random-walk MH against unconstrained random-walk MH and NUTS at equal likelihood evaluations. Measure predictive RMSE or accuracy, uncertainty calibration, effective sample size per evaluation, integrated autocorrelation time, rejection rate, and numerical divergences. Success means higher effective samples and fewer unstable trajectories, especially under artificially heavy-tailed priors; boundary concentration near $\|y\|=1$ is a falsifiable failure mode.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.