✗ Failed on benchmark 2026

Dynamic-programming Doob sampler for exact rare-event conditioning

Implementation & benchmark of arXiv:2608.17511 — Conditional-path Monte Carlo for rare stochastic dynamics on networks: Details and derivations

Usefulness7/10
Difficulty6/10
Novelty5/10

Source paper: Conditional-path Monte Carlo for rare stochastic dynamics on networks: Details and derivations arXiv:2608.17511 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Add an exact backward-conditioning module to a neural state-space model so trajectories satisfy a terminal label, target set, initial-state restriction, or prescribed event count without rejection. The module computes a backward feasibility message and reweights each neural transition toward states that can still satisfy the constraint, producing a conditioned process equivalent to a Doob transform. For large latent spaces, the exact message can be approximated by a value network and its approximation error can be measured directly.

Formulas

$$h_T(z)=\mathbf{1}_B(z),\qquad h_t(z)=\sum_{z'}P_\theta(z'\mid z)h_{t+1}(z').$$
$$P^C_\theta(z'\mid z,t)=\frac{P_\theta(z'\mid z)h_{t+1}(z')}{h_t(z)},\qquad \sum_{z'}P^C_\theta(z'\mid z,t)=1.$$$
$$h_t(z,c)=\sum_{z'}P_\theta(z'\mid z)h_{t+1}\!\left(z',c+g(z,z')\right),\qquad h_T(z,c)=\mathbf{1}\{c=m\}.$$$
$$\Pr_\theta(\omega\mid C)=\prod_{t=0}^{T-1}P^C_\theta(z_{t+1}\mid z_t,t).$$

Mathematical statement

Let $P_\theta(z'\mid z)$ be a discrete-time neural transition kernel and let $C(z_{0:T})\in\{0,1\}$ denote the desired path constraint. Define the backward message $h_t(z)=\Pr_\theta(C=1\mid z_t=z)$, with terminal condition $h_T(z)=\mathbf{1}_B(z)$ for terminal set $B$. The conditioned transition is $P^C_\theta(z'\mid z,t)=P_\theta(z'\mid z)h_{t+1}(z')/h_t(z)$ whenever $h_t(z)>0$; it is normalized by the backward recursion. For an event-count constraint, augment the state by count $c$, let $g(z,z')$ be the increment, and recursively compute $h_t(z,c)$. Exact messages give exact samples from the original trajectory law conditioned on $C$; approximate messages give a biased sampler whose KL error can be tested.

Implementation notes

1. Exact integration point: place the module between a neural transition model and its rollout sampler. The base model may be an SSM, GRU, transformer world model, or discrete diffusion transition model with normalized kernel $P_\theta$. The target can be a terminal class, reachability set, or exact number $m$ of threshold crossings. 2. Pseudocode: initialize terminal values $h_T$ from the constraint; for $t=T-1$ down to zero, compute $h_t(z)$ by summing over candidate next states, or use dynamic programming over a discretized latent state and event count; at rollout time sample $z'$ from logits $\log P_\theta(z'|z)+\log h_{t+1}(z')-\log h_t(z)$; disallow states with zero feasibility. For large latent spaces, train a value network $v_\psi(z,t,c)\approx\log h_t(z,c)$ and use normalized logits $\log P_\theta(z'|z)+v_\psi(z',t+1,c')$. 3. Computed from the paper's mechanism: backward dynamic programming and rejection-free boundary enforcement. Estimated empirically: message tables or value-network estimates, normalization residuals, terminal-constraint violations, and KL divergence from exact conditioned paths. 4. First cheap experiment: construct a 10-state neural Markov model with horizon 50 and terminal-event probability $10^{-3}$. Compare rejection sampling, classifier-guided sampling, and the Doob sampler. Check terminal-event frequency, path log-likelihood, and effective sample size. The predicted signature is zero constraint violations for exact messages and a cost per accepted trajectory that remains approximately constant as the unconditioned event probability decreases, versus approximately $1/p_C$ forward rollouts for rejection sampling. For approximate messages, the KL gap to exact conditioned paths should decrease monotonically as the value-network message error decreases.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built an exact finite-state backward-message Doob sampler with rejection-free terminal conditioning and deterministic message-perturbation analysis. Backward probabilities matched binomial-tail predictions exactly, all sampled endpoints satisfied the target, kernels had zero measured normalization residual, and conditioned-path total variation was 0.038 on 20,000 samples. Approximate-message path KL scaled quadratically with perturbation, while Doob cost stayed at 9 transitions versus rejection cost growing to about 978.9 million draws for the rarest setting.

Agent confidence
9/10
Baseline
Rejection sampling mean draws/sample: 501.2, 49,733.7, 1,911,929.9, and 978,908,129.4 for q=0.5, 0.3, 0.2, 0.1; predicted 1/p: 512, 50,805, 1,953,125, and 1,000,000,000.
Idea
Exact Doob sampler: 9 transitions/sample for all tested rare-event settings, 0 terminal violations across horizons 9, 12, 20, 30, 50, maximum kernel normalization residual 0.0, path-law TV 0.038 at T=12, and KL/epsilon^2 approximately 0.317 for message perturbations epsilon 0.01-0.2.

Limitations: Only a small discrete monotone Markov chain was tested; no neural state-space model, learned value network, event-count augmentation, continuous or large latent state approximation, wall-clock benchmark, or classifier-guided sampler comparison was evaluated.

How to run: python3 doob_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.17511)

Significantly WORSE than the tuned baseline (+31414.5%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark
Terminal_reachability_mdp
Model
mlp_tiny
Paired seeds
8
Baseline mean
0
Idea mean
0
Effect (Δ)
0 (+31414.5%; negative = idea better)
Wins
0 / 8 paired seeds
p-value
0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect
±67178.2%
Mechanism
Not confirmed ✗
Practical effect
Hurts
Baseline tuning
swept over 3 configs
Limitations:

The registered benchmark uses a small finite-state terminal-reachability MDP and the intervention is implemented as exact-feasibility-weighted training visitation, not a full continuous latent trajectory sampler or learned value-network Doob transform. Rejection-sampling wall-clock cost, event-count conditioning, approximate-message KL at neural scale, and continuous pendulum rollout conditioning were not tested. The observed normalization residual is computed after explicit normalization and is not an independent learned-message accuracy test; terminal violations are the substantive mechanism measurement.

How to run: python3 doob_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark latest Failed ✗

Agent confidence: 10/10

The full registered benchmark protocol was run on terminal_reachability_mdp with eight paired seeds, a tuned baseline sweep, and three shared learning rates for the idea. The exact Bellman/Doob check passed, but the idea was significantly worse on the independent test MSE (paired delta +3.32995e-06, permutation p=0.0081), so worked=false. The learned-message mechanism signature was also not confirmed because terminal violation rate was 5.8%.

Baseline
Best lr=0.003; full 8-seed mean test MSE=1.0600038902186526e-08.
Idea
Best shared-grid lr=0.01; full 8-seed mean test MSE=3.3405517725348943e-06; paired delta=+3.329951733632708e-06; p=0.0081; 0/8 paired wins.

Benchmark result

Significantly WORSE than the tuned baseline (+31414.5%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark
Terminal_reachability_mdp
Model
mlp_tiny
Paired seeds
8
Baseline mean
0
Idea mean
0
Effect (Δ)
0 (+31414.5%; negative = idea better)
Wins
0 / 8 paired seeds
p-value
0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect
±67178.2%
Mechanism
Not confirmed ✗
Practical effect
Hurts
Baseline tuning
swept over 3 configs
Limitations:

The registered benchmark uses a small finite-state terminal-reachability MDP and the intervention is implemented as exact-feasibility-weighted training visitation, not a full continuous latent trajectory sampler or learned value-network Doob transform. Rejection-sampling wall-clock cost, event-count conditioning, approximate-message KL at neural scale, and continuous pendulum rollout conditioning were not tested. The observed normalization residual is computed after explicit normalization and is not an independent learned-message accuracy test; terminal violations are the substantive mechanism measurement.

How to run: python3 doob_bench.py

Artifacts

Implementation overview ⬇ Download all as ZIP 8 files · code, reports and structured results