Dynamic-programming Doob sampler for exact rare-event conditioning
Implementation & benchmark of arXiv:2608.17511 — Conditional-path Monte Carlo for rare stochastic dynamics on networks: Details and derivations
Source paper: Conditional-path Monte Carlo for rare stochastic dynamics on networks: Details and derivations arXiv:2608.17511 ⓘ · analyzed Sep 1, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Add an exact backward-conditioning module to a neural state-space model so trajectories satisfy a terminal label, target set, initial-state restriction, or prescribed event count without rejection. The module computes a backward feasibility message and reweights each neural transition toward states that can still satisfy the constraint, producing a conditioned process equivalent to a Doob transform. For large latent spaces, the exact message can be approximated by a value network and its approximation error can be measured directly.
Formulas
Mathematical statement
Let $P_\theta(z'\mid z)$ be a discrete-time neural transition kernel and let $C(z_{0:T})\in\{0,1\}$ denote the desired path constraint. Define the backward message $h_t(z)=\Pr_\theta(C=1\mid z_t=z)$, with terminal condition $h_T(z)=\mathbf{1}_B(z)$ for terminal set $B$. The conditioned transition is $P^C_\theta(z'\mid z,t)=P_\theta(z'\mid z)h_{t+1}(z')/h_t(z)$ whenever $h_t(z)>0$; it is normalized by the backward recursion. For an event-count constraint, augment the state by count $c$, let $g(z,z')$ be the increment, and recursively compute $h_t(z,c)$. Exact messages give exact samples from the original trajectory law conditioned on $C$; approximate messages give a biased sampler whose KL error can be tested.
Implementation notes
1. Exact integration point: place the module between a neural transition model and its rollout sampler. The base model may be an SSM, GRU, transformer world model, or discrete diffusion transition model with normalized kernel $P_\theta$. The target can be a terminal class, reachability set, or exact number $m$ of threshold crossings. 2. Pseudocode: initialize terminal values $h_T$ from the constraint; for $t=T-1$ down to zero, compute $h_t(z)$ by summing over candidate next states, or use dynamic programming over a discretized latent state and event count; at rollout time sample $z'$ from logits $\log P_\theta(z'|z)+\log h_{t+1}(z')-\log h_t(z)$; disallow states with zero feasibility. For large latent spaces, train a value network $v_\psi(z,t,c)\approx\log h_t(z,c)$ and use normalized logits $\log P_\theta(z'|z)+v_\psi(z',t+1,c')$. 3. Computed from the paper's mechanism: backward dynamic programming and rejection-free boundary enforcement. Estimated empirically: message tables or value-network estimates, normalization residuals, terminal-constraint violations, and KL divergence from exact conditioned paths. 4. First cheap experiment: construct a 10-state neural Markov model with horizon 50 and terminal-event probability $10^{-3}$. Compare rejection sampling, classifier-guided sampling, and the Doob sampler. Check terminal-event frequency, path log-likelihood, and effective sample size. The predicted signature is zero constraint violations for exact messages and a cost per accepted trajectory that remains approximately constant as the unconditioned event probability decreases, versus approximately $1/p_C$ forward rollouts for rejection sampling. For approximate messages, the KL gap to exact conditioned paths should decrease monotonically as the value-network message error decreases.
Verification
Stage 1 · Toy mechanism gate: Passed ✓
Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗
Stage 2 · Practical benchmark result: Significantly worse ✗
Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works
Stage 1 — Mechanism check agent confidence 9/10
Built an exact finite-state backward-message Doob sampler with rejection-free terminal conditioning and deterministic message-perturbation analysis. Backward probabilities matched binomial-tail predictions exactly, all sampled endpoints satisfied the target, kernels had zero measured normalization residual, and conditioned-path total variation was 0.038 on 20,000 samples. Approximate-message path KL scaled quadratically with perturbation, while Doob cost stayed at 9 transitions versus rejection cost growing to about 978.9 million draws for the rarest setting.
- Agent confidence
- 9/10
- Baseline
- Rejection sampling mean draws/sample: 501.2, 49,733.7, 1,911,929.9, and 978,908,129.4 for q=0.5, 0.3, 0.2, 0.1; predicted 1/p: 512, 50,805, 1,953,125, and 1,000,000,000.
- Idea
- Exact Doob sampler: 9 transitions/sample for all tested rare-event settings, 0 terminal violations across horizons 9, 12, 20, 30, 50, maximum kernel normalization residual 0.0, path-law TV 0.038 at T=12, and KL/epsilon^2 approximately 0.317 for message perturbations epsilon 0.01-0.2.
Limitations: Only a small discrete monotone Markov chain was tested; no neural state-space model, learned value network, event-count augmentation, continuous or large latent state approximation, wall-clock benchmark, or classifier-guided sampler comparison was evaluated.
How to run: python3 doob_experiment.py
Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.17511)
Significantly WORSE than the tuned baseline (+31414.5%, p=0.0081, wins 0 / 8 paired seeds).
- Benchmark
- Terminal_reachability_mdp
- Model
- mlp_tiny
- Paired seeds
- 8
- Baseline mean
- 0
- Idea mean
- 0
- Effect (Δ)
- 0 (+31414.5%; negative = idea better)
- Wins
- 0 / 8 paired seeds
- p-value
- 0.0081 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- ±67178.2%
- Mechanism
- Not confirmed ✗
- Practical effect
- Hurts
- Baseline tuning
- swept over 3 configs
The registered benchmark uses a small finite-state terminal-reachability MDP and the intervention is implemented as exact-feasibility-weighted training visitation, not a full continuous latent trajectory sampler or learned value-network Doob transform. Rejection-sampling wall-clock cost, event-count conditioning, approximate-message KL at neural scale, and continuous pendulum rollout conditioning were not tested. The observed normalization residual is computed after explicit normalization and is not an independent learned-message accuracy test; terminal violations are the substantive mechanism measurement.
How to run: python3 doob_bench.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Stage 2 — Benchmark latest Failed ✗
Agent confidence: 10/10
The full registered benchmark protocol was run on terminal_reachability_mdp with eight paired seeds, a tuned baseline sweep, and three shared learning rates for the idea. The exact Bellman/Doob check passed, but the idea was significantly worse on the independent test MSE (paired delta +3.32995e-06, permutation p=0.0081), so worked=false. The learned-message mechanism signature was also not confirmed because terminal violation rate was 5.8%.
- Baseline
- Best lr=0.003; full 8-seed mean test MSE=1.0600038902186526e-08.
- Idea
- Best shared-grid lr=0.01; full 8-seed mean test MSE=3.3405517725348943e-06; paired delta=+3.329951733632708e-06; p=0.0081; 0/8 paired wins.
Benchmark result
Significantly WORSE than the tuned baseline (+31414.5%, p=0.0081, wins 0 / 8 paired seeds).
- Benchmark
- Terminal_reachability_mdp
- Model
- mlp_tiny
- Paired seeds
- 8
- Baseline mean
- 0
- Idea mean
- 0
- Effect (Δ)
- 0 (+31414.5%; negative = idea better)
- Wins
- 0 / 8 paired seeds
- p-value
- 0.0081 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- ±67178.2%
- Mechanism
- Not confirmed ✗
- Practical effect
- Hurts
- Baseline tuning
- swept over 3 configs
The registered benchmark uses a small finite-state terminal-reachability MDP and the intervention is implemented as exact-feasibility-weighted training visitation, not a full continuous latent trajectory sampler or learned value-network Doob transform. Rejection-sampling wall-clock cost, event-count conditioning, approximate-message KL at neural scale, and continuous pendulum rollout conditioning were not tested. The observed normalization residual is computed after explicit normalization and is not an independent learned-message accuracy test; terminal violations are the substantive mechanism measurement.
How to run: python3 doob_bench.py
Artifacts
- bench_report.json 4.3 KB View Raw
- doob_bench.py 7.3 KB View Raw
- doob_experiment.py 5.9 KB View Raw
- math_check.json 0.1 KB View Raw
- report.md 1.6 KB View
- report_bench_2026-09-02T235101.md 6.4 KB View
- results.json 3.1 KB View Raw
- stage2_bench.py 3.6 KB View Raw