# ESS-Controlled Autoregressive CTMC Sampler

- ID: 3114
- Canonical URL: https://synthcore.org/idea/3114/ess-controlled-autoregressive-ctmc-sampler
- API JSON: https://synthcore.org/api/idea/3114.json
- API Markdown: https://synthcore.org/api/idea/3114.md
- Verification status: unverified
- Source: [arXiv:2609.02948](https://arxiv.org/abs/2609.02948)
- Category: sampling
- Solves: sample-efficiency, stability, accuracy
- ML areas: diffusion, sampling, training
- Math tags: probability, stochastic-processes, statistics, combinatorics
- Ratings: usefulness 7/10; difficulty 6/10; novelty 7/10

## Idea description

Generate discrete configurations globally with an autoregressive model and then refine them using a continuous-time Markov chain of local single-site replacement moves. Use importance weights and the paper's normalized ESS to adapt the CTMC refinement budget and to reject training batches in which the proposal has collapsed onto a few modes.

## Mathematical statement

The extracted paper defines configurations \(\sigma\), a chemical or task condition \(c\), an initial distribution \(\rho_0\), and a target Boltzmann distribution \(\rho_1=p_c\). A CTMC rate \(Q_t^\theta(\tau,i\mid\sigma)\ge 0\) proposes replacing the species at site \(i\) by \(\tau\ne\sigma_i\), producing \(\sigma'=\operatorname{Swap}(\sigma,i,\tau)\). The paper's explicit diagnostic is \(\operatorname{ESS}=((\sum_{m=1}^M e^{A^{(m)}})^2)/(M\sum_{m=1}^M e^{2A^{(m)}})\in(1/M,1]\), where \(A^{(m)}\) is the log importance weight of sample \(m\), and \(M\operatorname{ESS}\) is the estimated number of equally useful independent samples. For a neural adaptation, let \(q_\phi(\sigma\mid c)\) be an autoregressive proposal, \(E_c(\sigma)\) a computable energy or negative reward, and \(\beta\) an inverse-temperature or scale parameter. Define \(p_c(\sigma)\propto\exp[-\beta E_c(\sigma)]\) and the initial importance log-weight \(A_0(\sigma)= -\beta E_c(\sigma)-\log q_\phi(\sigma\mid c)\). During CTMC refinement, use nonnegative local rates \(r_\theta(\tau,i\mid\sigma,c)\), total exit rate \(\lambda_\theta(\sigma)=\sum_{i,\tau\ne\sigma_i}r_\theta(\tau,i\mid\sigma,c)\), and generator entries \(Q_\theta(\sigma',\sigma)=r_\theta(\tau,i\mid\sigma,c)\) for a valid one-site replacement and \(Q_\theta(\sigma,\sigma)=-\lambda_\theta(\sigma)\). Estimate the CTMC path likelihood for jumps \((\sigma_{k-1},\sigma_k)\) at times \(t_k\) as \(\log P_\theta(\mathrm{path}\mid\sigma_0)=\sum_k\log r_\theta(\sigma_k\mid\sigma_{k-1})-\int_0^T\lambda_\theta(\sigma_t)\,dt\). Use \(A=A_0-\log P_\theta(\mathrm{path}\mid\sigma_0)\) as an importance-weight proxy when comparing generated paths, or initially treat the CTMC as a fixed augmentation and compute ESS from \(A_0\) only.

## Key formulas

- $$\mathrm{ESS}=\frac{\left(\sum_{m=1}^{M}e^{A^{(m)}}\right)^2}{M\sum_{m=1}^{M}e^{2A^{(m)}}}\in\left(\frac{1}{M},1\right],$$
- $$\sigma'={\rm Swap}(\sigma,i,\tau),\qquad Q_\theta(\sigma',\sigma)=r_\theta(\tau,i\mid\sigma,c)\ge 0,\qquad Q_\theta(\sigma,\sigma)=-\lambda_\theta(\sigma),$$
$$\lambda_\theta(\sigma)=\sum_{i=1}^{L}\sum_{\tau\ne\sigma_i}r_\theta(\tau,i\mid\sigma,c),$$
- $$A_0^{(m)}=-\beta E_c(\sigma_0^{(m)})-\log q_\phi(\sigma_0^{(m)}\mid c),\qquad \widetilde w_m=\exp\left(A^{(m)}-\operatorname{LSE}(A^{(1)},\ldots,A^{(M)})\right),$$
- $$\mathcal L(\phi,\theta)=-\sum_{m=1}^{M}\widetilde w_m\log q_\phi(\sigma_0^{(m)}\mid c)+\lambda_{\rm ESS}\,[\max(0,\eta-\mathrm{ESS})]^2$$

## Implementation notes

(1) Integration point: implement this as a sampler module for discrete sequences of length \(L\), categorical assignments, or alloy-like site labels. The autoregressive model \(q_\phi\) produces \(\sigma_0\) one site at a time. Apply a CTMC refinement kernel after generation, using a small number of event-driven jumps; each event changes exactly one categorical site. A network head outputs logits for every allowed replacement, and rates are \(r_\theta=\operatorname{softplus}(g_\theta(\sigma,c))\). (2) Pseudocode: sample a batch of conditions \(c\); sample \(\sigma_0^{(m)}\sim q_\phi(\cdot\mid c)\); set \(t=0\); while \(t<T\), compute all local rates, set \(\lambda=\sum r\), draw waiting time \(\Delta t\sim\operatorname{Exponential}(\lambda)\), choose move \((i,\tau)\) with probability \(r(\tau,i)/\lambda\), replace \(\sigma_i\), and accumulate \(\log r-\lambda\Delta t\); calculate \(A_0=-\beta E-\log q_\phi\), optionally set \(A=A_0-\log P_\theta(\mathrm{path})\), normalize weights with log-sum-exp, compute the displayed ESS, and optimize the weighted autoregressive loss plus the ESS penalty. Use \(\eta=0.2\) initially. (3) Compute from the mathematics: the local generator, exponential waiting times, categorical jump selection, log path likelihood, normalized weights, and ESS. Estimate empirically: energy scale \(\beta\), refinement horizon \(T\), target ESS \(\eta\), and whether including the CTMC path correction improves calibration. For a first stable version, stop gradients through ESS and use it only for adaptive control: increase \(T\) when ESS is low and decrease it when ESS is high. (4) First experiment: use a length-32 binary Ising chain or a length-32 categorical sequence with a known energy, comparing an autoregressive sampler, autoregressive sampling followed by uniformly random single-site mutations, and the learned-rate CTMC sampler. Train with batches of 128 configurations and evaluate ESS, unique-mode count, energy-distribution error, autocorrelation, and wall-clock cost. Success means higher ESS and more recovered low-probability modes at equal model evaluations, with no degradation in mean energy or target-distribution KL estimated against exact enumeration on small lengths.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.