Geometry-Consistent Latent Particle Rollouts
Implementation & benchmark of arXiv:2607.17781 — Geometry-Consistent Bayesian Filtering under Structural Model Uncertainty: A Geometric Projection Particle Filter
Source paper: Geometry-Consistent Bayesian Filtering under Structural Model Uncertainty: A Geometric Projection Particle Filter arXiv:2607.17781 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Use the observation Jacobian to remove from a neural latent dynamics model the component of its drift that is locally inconsistent with the observed manifold. Apply this projected drift only to generate particle proposals, and retain exact importance-ratio correction so that proposal projection improves particle coverage without changing the target posterior.
Formulas
Mathematical statement
Let the learned latent dynamics be $dx=f_\theta(x,t)dt+G_\theta(x,t)dB_t$, and let the decoder or observation head be $y=h_\phi(x,t)+\epsilon$ with Jacobian $H=\partial h_\phi/\partial x$ and observation covariance $R$. For a locally constant observation, the tangent projector is $P=I-H^\top(HH^\top+\tau I)^{-1}H$, where $\tau>0$ stabilizes rank-deficient Jacobians. More generally, with desired observation velocity $\dot y$, define the geometry-consistent drift as the minimum-change solution of $\min_v (v-f_\theta)^\top Q^{-1}(v-f_\theta)$ subject to $Hv=\dot y$, giving $f_{gc}=f_\theta+QH^\top(HQH^\top)^{-1}(\dot y-Hf_\theta)$; the soft version replaces the inverse by $(HQH^\top+\tau R)^{-1}$. The geometric co-state $\lambda=H^\top R^{-1}(Hf_\theta-\dot y)$ measures drift-measurement incompatibility. For structural model error $\delta f$, only $P\delta f$ remains after projection, predicting robustness controlled by orthogonal mismatch rather than total mismatch.
Implementation notes
1. Integration point: implement this inside a latent state-space model or neural ODE used for partially observed rollout, planning, or filtering. The neural dynamics network outputs $f_\theta(x_k)$ and optionally a diagonal diffusion $G_\theta(x_k)$; a decoder outputs $h_\phi(x_k)$ and its Jacobian with respect to the latent state. Do not modify the training target initially: use the projection only for inference-time particle proposals. 2. Pseudocode: for every particle, compute $H=J_xh_\phi(x_k)$, estimate $\dot y=(y_{k+1}-y_k)/\Delta t$ or set $\dot y=0$ for static observations, compute $f_{gc}=f_\theta+QH^\top(HQH^\top+\tau R)^{-1}(\dot y-Hf_\theta)$, sample $x_{k+1}$ from the Gaussian proposal, evaluate the nominal transition density, proposal density, and observation likelihood, then update the log weight with the displayed importance ratio. Resample only when ESS $=(\sum_iw_i)^2/\sum_iw_i^2$ falls below $0.5N$. 3. Computed from the mechanism: the projection, co-state, proposal density, and importance correction. Estimated empirically: Jacobians by autodiff, $Q$ from the learned diffusion or a diagonal tunable matrix, and $\dot y$ by finite differences. 4. First cheap experiment: use a 4D Lorenz-63 or Lorenz-96 system with only two observed coordinates, train a small MLP latent dynamics model on nominal trajectories, and test filtering under injected drift perturbations. Compare bootstrap PF, projected PF without correction, and geometry-consistent PF with correction at equal particle count. The key falsifiable signature is that RMSE and ESS degradation under perturbation correlate with $\|P\delta f\|$, not $\|\delta f\|$: perturbations parallel to the rows of $H$ should be substantially suppressed, while tangent-space perturbations should remain. Across perturbation directions, regress squared filtering error against both quantities; the $\|P\delta f\|^2$ model should explain more variance. Also verify $\|Hf_{gc}-\dot y\|$ decreases approximately monotonically as $\tau$ decreases until numerical conditioning fails, and that corrected and unprojected filters have statistically indistinguishable posterior means when proposal variance is sufficiently large.
Verification
Stage 1 · Toy mechanism gate: Passed ✓
Stage 2 · Mechanism transferred to benchmark: Confirmed ✓
Stage 2 · Practical benchmark result: Significantly worse ✗
Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works
Stage 1 — Mechanism check agent confidence 7/10
Built a NumPy geometry-consistent particle proposal with soft Jacobian correction, proposal-density importance correction, ESS-triggered resampling, and reproducible sweeps. Three mechanism predictions were confirmed: the soft observation residual matched its closed form to 3.5e-11 relative error and increased monotonically with tau; observed-direction projector attenuation matched tau/(1+tau) within 1.1e-16; and tangent directions were preserved exactly. However, the corrected filter had RMSE 0.515 versus 0.415 for bootstrap and mean ESS 35.3 versus 153.7, while the directional sweep explained only R2=0.071 with projected mismatch, so no practical filtering win was observed.
- Agent confidence
- 7/10
- Baseline
- Bootstrap PF: RMSE 0.4155, mean ESS 153.7
- Idea
- Geometry-consistent PF with exact importance correction: RMSE 0.5151, mean ESS 35.35; soft residual identity relative error 3.48e-11; radial attenuation max error 1.11e-16; tangent preservation max error 0
Limitations: This is a linear 4D toy model with a fixed observation Jacobian, not a learned neural decoder or Lorenz system. Only one main filtering trajectory and a small eight-direction sweep were tested; proposal variance tuning, multiple seeds, nonlinear observations, and statistical confidence intervals were not evaluated.
How to run: python3 geometry_pf_experiment.py
Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.17781)
Significantly WORSE than the tuned baseline (+0.6%, p=0.0081, wins 0 / 8 paired seeds).
- Benchmark
- Dynamics forecasting
- Model
- rnn_small
- Paired seeds
- 8
- Baseline mean
- 1.5443
- Idea mean
- 1.5535
- Effect (Δ)
- 0.0092 (+0.6%; negative = idea better)
- Wins
- 0 / 8 paired seeds
- p-value
- 0.0081 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- ±0.1%
- Mechanism
- Confirmed ✓
- Practical effect
- Hurts
- Baseline tuning
- swept over 3 configs
The fixed bench exposes supervised pendulum window-to-angle prediction rather than a full particle filter with stochastic latent diffusion, explicit observation likelihood, and importance-ratio correction. The implementation therefore tests transfer of the Jacobian-consistent latent rollout correction, not the complete particle proposal algorithm; no larger models, longer horizons, alternate observation dimensions, or particle-count sweeps were tested.
How to run: python3 bench_stage2.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Stage 2 — Benchmark latest Failed ✗
Agent confidence: 10/10
Implemented the geometry-consistent latent drift correction on the structurally matched controlled-pendulum dynamics track using identical independently trained GRUs and canonical train_model training. The trained-model mechanism signature was confirmed: mean |Hf-ydot| fell from 0.6601 to 0.1040, an 84.2% reduction. However, test MSE worsened from 1.5443 to 1.5535, paired delta +0.00921 with permutation p=0.0081; therefore the mechanism works algebraically at neural-network scale but does not improve the benchmark task metric.
- Baseline
- Dynamics rnn_small, tuned lr=0.001: test MSE 1.544309 ± 0.042309 over 8 seeds; baseline sweep means lr=0.001: 1.555335, lr=0.003: 1.560285, lr=0.006: 1.558652.
- Idea
- Geometry-consistent latent rollout, best tested cfg lr=0.001, tau=0.1: test MSE 1.553522 ± 0.043264 over 8 seeds; paired delta +0.009212, 0/8 wins, permutation p=0.0081. Mechanism residual reduced 84.24% (0.660070 to 0.104020).
Benchmark result
Significantly WORSE than the tuned baseline (+0.6%, p=0.0081, wins 0 / 8 paired seeds).
- Benchmark
- Dynamics forecasting
- Model
- rnn_small
- Paired seeds
- 8
- Baseline mean
- 1.5443
- Idea mean
- 1.5535
- Effect (Δ)
- 0.0092 (+0.6%; negative = idea better)
- Wins
- 0 / 8 paired seeds
- p-value
- 0.0081 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- ±0.1%
- Mechanism
- Confirmed ✓
- Practical effect
- Hurts
- Baseline tuning
- swept over 3 configs
The fixed bench exposes supervised pendulum window-to-angle prediction rather than a full particle filter with stochastic latent diffusion, explicit observation likelihood, and importance-ratio correction. The implementation therefore tests transfer of the Jacobian-consistent latent rollout correction, not the complete particle proposal algorithm; no larger models, longer horizons, alternate observation dimensions, or particle-count sweeps were tested.
How to run: python3 bench_stage2.py
Artifacts
- bench_report.json 4.3 KB View Raw
- bench_stage2.py 5.1 KB View Raw
- geometry_pf_experiment.py 7.1 KB View Raw
- report.md 1.6 KB View
- report_bench_2026-08-30T195503.md 4.7 KB View
- results.json 2.9 KB View Raw