△ Mechanism confirmed, baseline not beaten 2026

Geometry-Consistent Latent Particle Rollouts

Implementation & benchmark of arXiv:2607.17781 — Geometry-Consistent Bayesian Filtering under Structural Model Uncertainty: A Geometric Projection Particle Filter

Usefulness8/10
Difficulty6/10
Novelty7/10

Source paper: Geometry-Consistent Bayesian Filtering under Structural Model Uncertainty: A Geometric Projection Particle Filter arXiv:2607.17781 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Use the observation Jacobian to remove from a neural latent dynamics model the component of its drift that is locally inconsistent with the observed manifold. Apply this projected drift only to generate particle proposals, and retain exact importance-ratio correction so that proposal projection improves particle coverage without changing the target posterior.

Formulas

$$P=I-H^\top(HH^\top+\tau I)^{-1}H,\qquad HP\approx 0.$$
$$f_{gc}=f_\theta+QH^\top(HQH^\top+\tau R)^{-1}(\dot y-Hf_\theta),\qquad \lambda=H^\top R^{-1}(Hf_\theta-\dot y).$$
$$x_{k+1}^{(i)}\sim q_{gc}(\cdot\mid x_k^{(i)},y_{k+1})=\mathcal N\left(x_k^{(i)}+f_{gc}(x_k^{(i)})\Delta t,\;G_\theta G_\theta^\top\Delta t\right).$$
$$w_{k+1}^{(i)}\propto w_k^{(i)}\,p_\theta(x_{k+1}^{(i)}\mid x_k^{(i)})\,p_\phi(y_{k+1}\mid x_{k+1}^{(i)})/q_{gc}(x_{k+1}^{(i)}\mid x_k^{(i)},y_{k+1}).$$

Mathematical statement

Let the learned latent dynamics be $dx=f_\theta(x,t)dt+G_\theta(x,t)dB_t$, and let the decoder or observation head be $y=h_\phi(x,t)+\epsilon$ with Jacobian $H=\partial h_\phi/\partial x$ and observation covariance $R$. For a locally constant observation, the tangent projector is $P=I-H^\top(HH^\top+\tau I)^{-1}H$, where $\tau>0$ stabilizes rank-deficient Jacobians. More generally, with desired observation velocity $\dot y$, define the geometry-consistent drift as the minimum-change solution of $\min_v (v-f_\theta)^\top Q^{-1}(v-f_\theta)$ subject to $Hv=\dot y$, giving $f_{gc}=f_\theta+QH^\top(HQH^\top)^{-1}(\dot y-Hf_\theta)$; the soft version replaces the inverse by $(HQH^\top+\tau R)^{-1}$. The geometric co-state $\lambda=H^\top R^{-1}(Hf_\theta-\dot y)$ measures drift-measurement incompatibility. For structural model error $\delta f$, only $P\delta f$ remains after projection, predicting robustness controlled by orthogonal mismatch rather than total mismatch.

Implementation notes

1. Integration point: implement this inside a latent state-space model or neural ODE used for partially observed rollout, planning, or filtering. The neural dynamics network outputs $f_\theta(x_k)$ and optionally a diagonal diffusion $G_\theta(x_k)$; a decoder outputs $h_\phi(x_k)$ and its Jacobian with respect to the latent state. Do not modify the training target initially: use the projection only for inference-time particle proposals. 2. Pseudocode: for every particle, compute $H=J_xh_\phi(x_k)$, estimate $\dot y=(y_{k+1}-y_k)/\Delta t$ or set $\dot y=0$ for static observations, compute $f_{gc}=f_\theta+QH^\top(HQH^\top+\tau R)^{-1}(\dot y-Hf_\theta)$, sample $x_{k+1}$ from the Gaussian proposal, evaluate the nominal transition density, proposal density, and observation likelihood, then update the log weight with the displayed importance ratio. Resample only when ESS $=(\sum_iw_i)^2/\sum_iw_i^2$ falls below $0.5N$. 3. Computed from the mechanism: the projection, co-state, proposal density, and importance correction. Estimated empirically: Jacobians by autodiff, $Q$ from the learned diffusion or a diagonal tunable matrix, and $\dot y$ by finite differences. 4. First cheap experiment: use a 4D Lorenz-63 or Lorenz-96 system with only two observed coordinates, train a small MLP latent dynamics model on nominal trajectories, and test filtering under injected drift perturbations. Compare bootstrap PF, projected PF without correction, and geometry-consistent PF with correction at equal particle count. The key falsifiable signature is that RMSE and ESS degradation under perturbation correlate with $\|P\delta f\|$, not $\|\delta f\|$: perturbations parallel to the rows of $H$ should be substantially suppressed, while tangent-space perturbations should remain. Across perturbation directions, regress squared filtering error against both quantities; the $\|P\delta f\|^2$ model should explain more variance. Also verify $\|Hf_{gc}-\dot y\|$ decreases approximately monotonically as $\tau$ decreases until numerical conditioning fails, and that corrected and unprojected filters have statistically indistinguishable posterior means when proposal variance is sufficiently large.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 7/10

Built a NumPy geometry-consistent particle proposal with soft Jacobian correction, proposal-density importance correction, ESS-triggered resampling, and reproducible sweeps. Three mechanism predictions were confirmed: the soft observation residual matched its closed form to 3.5e-11 relative error and increased monotonically with tau; observed-direction projector attenuation matched tau/(1+tau) within 1.1e-16; and tangent directions were preserved exactly. However, the corrected filter had RMSE 0.515 versus 0.415 for bootstrap and mean ESS 35.3 versus 153.7, while the directional sweep explained only R2=0.071 with projected mismatch, so no practical filtering win was observed.

Agent confidence
7/10
Baseline
Bootstrap PF: RMSE 0.4155, mean ESS 153.7
Idea
Geometry-consistent PF with exact importance correction: RMSE 0.5151, mean ESS 35.35; soft residual identity relative error 3.48e-11; radial attenuation max error 1.11e-16; tangent preservation max error 0

Limitations: This is a linear 4D toy model with a fixed observation Jacobian, not a learned neural decoder or Lorenz system. Only one main filtering trajectory and a small eight-direction sweep were tested; proposal variance tuning, multiple seeds, nonlinear observations, and statistical confidence intervals were not evaluated.

How to run: python3 geometry_pf_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.17781)

Significantly WORSE than the tuned baseline (+0.6%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark
Dynamics forecasting
Model
rnn_small
Paired seeds
8
Baseline mean
1.5443
Idea mean
1.5535
Effect (Δ)
0.0092 (+0.6%; negative = idea better)
Wins
0 / 8 paired seeds
p-value
0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect
±0.1%
Mechanism
Confirmed ✓
Practical effect
Hurts
Baseline tuning
swept over 3 configs
Limitations:

The fixed bench exposes supervised pendulum window-to-angle prediction rather than a full particle filter with stochastic latent diffusion, explicit observation likelihood, and importance-ratio correction. The implementation therefore tests transfer of the Jacobian-consistent latent rollout correction, not the complete particle proposal algorithm; no larger models, longer horizons, alternate observation dimensions, or particle-count sweeps were tested.

How to run: python3 bench_stage2.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark latest Failed ✗

Agent confidence: 10/10

Implemented the geometry-consistent latent drift correction on the structurally matched controlled-pendulum dynamics track using identical independently trained GRUs and canonical train_model training. The trained-model mechanism signature was confirmed: mean |Hf-ydot| fell from 0.6601 to 0.1040, an 84.2% reduction. However, test MSE worsened from 1.5443 to 1.5535, paired delta +0.00921 with permutation p=0.0081; therefore the mechanism works algebraically at neural-network scale but does not improve the benchmark task metric.

Baseline
Dynamics rnn_small, tuned lr=0.001: test MSE 1.544309 ± 0.042309 over 8 seeds; baseline sweep means lr=0.001: 1.555335, lr=0.003: 1.560285, lr=0.006: 1.558652.
Idea
Geometry-consistent latent rollout, best tested cfg lr=0.001, tau=0.1: test MSE 1.553522 ± 0.043264 over 8 seeds; paired delta +0.009212, 0/8 wins, permutation p=0.0081. Mechanism residual reduced 84.24% (0.660070 to 0.104020).

Benchmark result

Significantly WORSE than the tuned baseline (+0.6%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark
Dynamics forecasting
Model
rnn_small
Paired seeds
8
Baseline mean
1.5443
Idea mean
1.5535
Effect (Δ)
0.0092 (+0.6%; negative = idea better)
Wins
0 / 8 paired seeds
p-value
0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect
±0.1%
Mechanism
Confirmed ✓
Practical effect
Hurts
Baseline tuning
swept over 3 configs
Limitations:

The fixed bench exposes supervised pendulum window-to-angle prediction rather than a full particle filter with stochastic latent diffusion, explicit observation likelihood, and importance-ratio correction. The implementation therefore tests transfer of the Jacobian-consistent latent rollout correction, not the complete particle proposal algorithm; no larger models, longer horizons, alternate observation dimensions, or particle-count sweeps were tested.

How to run: python3 bench_stage2.py

Artifacts

Implementation overview ⬇ Download all as ZIP 6 files · code, reports and structured results