# Path-work correction for exact neural proposals

- ID: 974
- Canonical URL: https://synthcore.org/idea/974/path-work-correction-for-exact-neural-proposals
- API JSON: https://synthcore.org/api/idea/974.json
- API Markdown: https://synthcore.org/api/idea/974.md
- Verification status: beats_baseline
- Source: [arXiv:2607.15682](https://arxiv.org/abs/2607.15682)
- Category: sampling
- Solves: accuracy, stability, sample-efficiency
- ML areas: diffusion-sampling, inference-speedup, training
- Math tags: probability, stochastic-processes, measure-theory, information-theory
- Ratings: usefulness 8/10; difficulty 4/10; novelty 6/10

## Idea description

Use the learned path only as a global proposal, then correct complete trajectories rather than endpoints. Exponentiated negative work gives self-normalized importance weights, while the same path ratio gives an independent Metropolis acceptance probability. This turns an imperfect neural sampler into an asymptotically exact sampler whenever forward and reverse path laws overlap.

## Mathematical statement

Let \(\tau^{(i)}\sim Q_\theta\) be independent complete paths, let \(W_i=W_\theta(\tau^{(i)})\) be their generalized work, and let \(x_i=x_T^{(i)}\) be endpoint configurations. Since \(\exp(-W_i)\) is proportional to the target-path density divided by the proposal-path density, path self-normalized importance sampling estimates an endpoint observable \(f\) by \(\widehat{\mathbb E_\pi[f]}=\sum_iw_if(x_i)/\sum_iw_i\), where \(w_i=\exp(-W_i)\). The effective sample size is \(\mathrm{ESS}=(\sum_iw_i)^2/\sum_iw_i^2\). For independent path Metropolis-Hastings, if \(\tau\) is the current path and \(\tau'\sim Q_\theta\) is a fresh proposal, the acceptance probability is \(\alpha(\tau\to\tau')=\min\{1,\exp[-W(\tau')+W(\tau)]\}\). The endpoint chain preserves the Boltzmann target after marginalizing intermediate states. A normalizer or free-energy ratio can be estimated using \(\Delta F=-\log\mathbb E_{Q_\theta}[\exp(-W_\theta)]\), relative to the selected base/reference free energy.

## Key formulas

- $$w_i=\exp[-W_\theta(\tau^{(i)})],\qquad \widehat{\mathbb E_\pi[f]}=\frac{\sum_{i=1}^{N}w_i f(x_T^{(i)})}{\sum_{i=1}^{N}w_i}$$
- $$\mathrm{ESS}=\frac{\left(\sum_{i=1}^{N}w_i\right)^2}{\sum_{i=1}^{N}w_i^2}$$
- $$\alpha(\tau\to\tau')=\min\left\{1,\exp\left[-W_\theta(\tau')+W_\theta(\tau)\right]\right\}$$
- $$\Delta F=-\log\mathbb{E}_{Q_\theta}\left[\exp(-W_\theta)\right]$$

## Implementation notes

Attach this correction after the neural Hamiltonian bridge has been trained and frozen. For path-SNIS, generate \(N\) complete trajectories, compute each forward path log probability, reverse-reference path log probability, and endpoint energy terms, then set \(\ell_i=-W_i\). For numerical stability subtract \(\max_i\ell_i\) before exponentiating: \(w_i=\exp(\ell_i-\max_j\ell_j)\). Normalize the weights for observables and log ESS/N, maximum normalized weight, and work variance. For path-IMH, retain the current complete path and its work, generate one independent trajectory, and replace the current path with probability \(\min(1,\exp(W_{\mathrm{old}}-W_{\mathrm{new}}))\), computed in log space. Return endpoint configurations while discarding intermediate states after each decision. The paper-derived calculations are the path ratio, importance weights, ESS, free-energy estimator, and acceptance ratio; autocorrelation and residual endpoint bias are empirical measurements. First test on a 2-D double well and a 10-D Gaussian mixture with 10,000 proposals, comparing raw neural endpoints, path-SNIS, path-IMH, random-walk Metropolis, and HMC at matched energy evaluations. Success means unbiased moments within Monte Carlo error, accurate mode probabilities, and lower integrated autocorrelation than local MCMC. Failure is ESS below 1% or near-zero IMH acceptance.

## Verification

- Status: beats_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: helps
- Verdict: Built a reproducible 1-D Gaussian path-correction MVP with complete-path records, stable path-SNIS, independent path-IMH, random-walk MH, and HMC baselines. For sigma=0.8, 1.3, 1.7, and 2.2, observed ESS fractions matched analytic predictions within 0.002, while observed IMH acceptance matched deterministic quadrature within 0.004. SNIS and IMH corrected the raw proposal second moment from 2.821 to 0.978 and 0.974 versus the target 1.0; IMH also had higher x² chain ESS than random-walk MH, 5215 versus 1443.

### Mechanism check

- Verdict: Built a reproducible 1-D Gaussian path-correction MVP with complete-path records, stable path-SNIS, independent path-IMH, random-walk MH, and HMC baselines. For sigma=0.8, 1.3, 1.7, and 2.2, observed ESS fractions matched analytic predictions within 0.002, while observed IMH acceptance matched deterministic quadrature within 0.004. SNIS and IMH corrected the raw proposal second moment from 2.821 to 0.978 and 0.974 versus the target 1.0; IMH also had higher x² chain ESS than random-walk MH, 5215 versus 1443.
- Confidence: 8/10
- Limitations: This is a normalized 1-D Gaussian toy, not the requested 2-D double well or 10-D Gaussian mixture, and auxiliary path coordinates have cancelling forward/reverse laws rather than a learned neural Hamiltonian bridge. Training, CUDA execution, FLOP-normalized timing, multimodal mode probabilities, and repeated confidence intervals were not tested.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 8/8
- Benchmark verdict: idea better (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/173/bench_report.json)
- [path_track.py](https://synthcore.org/code/173/path_track.py)
- [path_work_experiment.py](https://synthcore.org/code/173/path_work_experiment.py)
- [report.md](https://synthcore.org/code/173/report.md)
- [report_bench_2026-08-30T192843.md](https://synthcore.org/code/173/report_bench_2026-08-30T192843.md)
- [results.json](https://synthcore.org/code/173/results.json)
- [run_output.json](https://synthcore.org/code/173/run_output.json)
- [run_output2.json](https://synthcore.org/code/173/run_output2.json)
- [stage2_path_bench.py](https://synthcore.org/code/173/stage2_path_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/173)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.