✗ Failed on benchmark 2026

Sensitivity-Particle Training for Marginal-Only Latent ODEs

Implementation & benchmark of arXiv:2608.06190 — Characteristic Sensitivity Ensembles for Inference of Hidden Dynamics from Marginal Observations

Usefulness7/10

Difficulty6/10

Novelty7/10

Training Accuracy Sample efficiency Stability World model Training Optimizer Dynamical systems Probability Statistics Numerical analysis

Source paper: Characteristic Sensitivity Ensembles for Inference of Hidden Dynamics from Marginal Observations arXiv:2608.06190 ⓘ · analyzed Aug 31, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Train an augmented latent neural ODE from snapshot observations of only the visible coordinates by transporting particles from an initial latent distribution and differentiating their visible locations through forward sensitivity equations. Replace density-PDE discretization or potentially biased same-particle density objectives with a kernel marginal-matching loss whose gradient is estimated using independent particle sets.

Formulas

$$\dot{q}(t)=R(q(t),t;\alpha),\qquad q(0)=q_0,\qquad q=(x,y)\in\mathbb{R}^{d_x+d_y}.$$

$$\dot{S}(t)=J_R(q(t),t;\alpha)S(t)+\partial_\alpha R(q(t),t;\alpha),\qquad S(0)=\partial_\alpha q_0.$$

$$\hat p_\alpha(z,t)=\frac{1}{N}\sum_{i=1}^N K_h(z-X_i(t)),\qquad \partial_\alpha K_h(z-X_i)=-\nabla_zK_h(z-X_i)\,\pi_xS_i(t).$$

$$\widehat{\nabla_\alpha\mathcal{L}}=\frac{1}{|Z||A|}\sum_{z\in Z,i\in A}\nabla_\alpha K_h(z-X_i)-\frac{1}{|A||B|}\sum_{i\in A,j\in B}\nabla_\alpha K_h(X_i-X_j),\quad \mathcal{L}=\mathbb{E}K_h(Z-X)-\frac12\mathbb{E}K_h(X-X').$$

Mathematical statement

The paper starts from the generalized ODE $\dot{q}(t)=R(q(t),t;\alpha)$ with $q(0)=q_0$, where $q=(x,y)$ contains visible coordinates $x\in\mathbb{R}^{d_x}$ and hidden coordinates $y\in\mathbb{R}^{d_y}$, and $\alpha$ is the unknown parameter vector. Let $\Phi_t(q_0;\alpha)$ be the ODE flow, $Q_i(t)=\Phi_t(Q_i(0);\alpha)$ a simulated particle, and $X_i(t)=\pi_xQ_i(t)$ its visible projection. The forward sensitivity is $S_i(t)=\partial Q_i(t)/\partial\alpha\in\mathbb{R}^{d\times p}$ and satisfies $\dot{S}_i=J_R(Q_i,t;\alpha)S_i+\partial_\alpha R(Q_i,t;\alpha)$, with $S_i(0)=\partial_\alpha Q_i(0)$. For a kernel $K_h(u)=h^{-d_x}K(u/h)$, the simulated marginal density is $\hat p_\alpha(z,t)=N^{-1}\sum_iK_h(z-X_i(t))$, and $\partial_\alpha K_h(z-X_i)=-\nabla_zK_h(z-X_i)\,\pi_xS_i$. A practical crossed estimator uses an observed minibatch $Z$ and independent simulated particle sets $A$ and $B$. The objective is $\mathcal{L}=\mathbb{E}[K_h(Z-X)]-\tfrac{1}{2}\mathbb{E}[K_h(X-X')]$, where $X$ and $X'$ are independent simulated visible particles. Its gradient estimator is $\widehat{\nabla_\alpha\mathcal{L}}=(|Z||A|)^{-1}\sum_{z\in Z,i\in A}\nabla_\alpha K_h(z-X_i)-(|A||B|)^{-1}\sum_{i\in A,j\in B}\nabla_\alpha K_h(X_i-X_j)$.

Implementation notes

Integrate this at the training step of a latent neural ODE or continuous-time world model. Let the neural vector field be $R_\theta(q,t)$, with $q=[x,y]$ and a low-dimensional latent state $y$. Initialize simulated particles by sampling visible initial states from the dataset and latent initial states from a learned diagonal Gaussian or an encoder. For each particle and each requested snapshot time, integrate the augmented state $(q_i,S_i)$ with an adaptive ODE solver. Obtain $J_R S_i$ and $\partial_\theta R$ using automatic-differentiation Jacobian-vector products rather than explicitly constructing a dense Jacobian. Split particles into independent sets A and B. For an observed minibatch Z, compute the Gaussian kernel $K_h(u)=(2\pi h^2)^{-d_x/2}\exp(-\lVert u\rVert^2/(2h^2))$, evaluate the two crossed sums in the gradient formula, and update $\theta$ with AdamW. Use bandwidth $h$ proportional to the median pairwise distance in the observed minibatch, clipped to a fixed range; resample latent initial states each iteration. The paper supplies the characteristic sensitivity recursion and unbiased crossed-estimator structure; bandwidth, particle count, solver tolerance, and latent prior are empirical settings. The first cheap experiment should use a two-dimensional bistable ODE with only the first coordinate observed, 64 particles per set, and a small MLP vector field. Compare against a same-particle KDE objective and a deterministic mean-field neural ODE. Evaluate held-out marginal kernel distance, mode locations, trajectory stability, and gradient variance. Success is lower held-out marginal error and smoother loss descent at equal particle count, particularly when the observed marginal is bimodal or observations are temporally sparse.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a reproducible sensitivity-particle KDE experiment with exact forward sensitivities, crossed independent-particle gradients, same-particle comparison, and equal-budget training. Finite-difference sensitivity error scaled as eps^2 (observed slope 1.9998 vs 2), crossed gradient variance scaled approximately as N^-1 (observed slope -0.9008), and the same-minus-cross difference showed 1/N scaling. Crossed training obtained a higher held-out objective than the same-particle baseline (0.3746 vs 0.3658), although this is only a small toy result.

Agent confidence: 8/10

Baseline: Same-particle KDE: held-out objective 0.36577; gradient variance decreased from 0.03407 at N=8 to 0.00281 at N=128; finite-N bias slope -0.914.
Idea: Crossed KDE: held-out objective 0.37461; gradient variance decreased from 0.03198 at N=8 to 0.00271 at N=128; variance slope -0.901 versus predicted -1. Sensitivity finite-difference slope was 1.9998 versus predicted 2.0.

Limitations: The MVP uses an analytically solvable scalar flow with bimodal particles rather than a learned MLP vector field, adaptive ODE integration, or a genuinely hidden latent coordinate. It does not test temporal sparsity, larger dimensions, adaptive bandwidths, CUDA execution, or multiple independent training seeds.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.06190) Sep 2, 2026

Significantly WORSE than the tuned baseline (+191472.4%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0007
Idea mean: 0
Effect (Δ): 1.2978 (+191472.4%; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±161065.6%
Mechanism: Not confirmed ✗
Practical effect: Hurts
Baseline tuning: swept over 3 configs

Limitations:

Only the built-in dynamics track was tested; the task is supervised pendulum rollout regression rather than snapshot-only latent marginal ODE learning. The implementation uses a fixed bandwidth and particle loss without adaptive ODE integration, explicit forward sensitivity equations, hidden latent-state initialization, temporal snapshots, or NN-scale gradient-variance scaling analysis.

How to run: python3 bench_experiment.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Failed ✗

Agent confidence: 10/10

Implemented crossed independent-particle Gaussian KDE training for the shared rnn_small dynamics model and compared it against standard MSE training under the required paired-seed protocol. Baseline test MSE was 0.000678, while the best idea configuration obtained 1.29845 mean MSE; paired delta was +1.29777 with permutation p=0.0081, indicating a significant degradation. The mechanism signature is not confirmed because NN-scale N^-1 variance/bias scaling was not re-tested.

Baseline: best_cfg lr=0.01; sweep means: lr=0.001 -> 0.003941, lr=0.003 -> 0.001289, lr=0.01 -> 0.000664; full 8-seed mean MSE=0.000677786
Idea: best_cfg lr=0.001; 8-seed mean MSE=1.298451; per-seed MSE=[0.376203, 2.215075, 0.431349, 0.589128, 0.077601, 3.351390, 0.448343, 2.898518]; paired delta=+1.297773; p=0.0081

Benchmark result