△ Mechanism confirmed, baseline not beaten 2026

Fisher-Identifiable Neural ODE Design

Implementation & benchmark of arXiv:2608.13044 — Identifiability-aware neural ordinary differential equations for parsimonious and reliable dynamic modelling

Usefulness8/10

Difficulty6/10

Novelty7/10

Dynamics Stability Generalization Accuracy Ssm Rnn Optimizer Regularization Training dynamics Dynamical systems Statistics Linear algebra Optimization

Source paper: Identifiability-aware neural ordinary differential equations for parsimonious and reliable dynamic modelling arXiv:2608.13044 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Train and select neural ODE architectures using parameter sensitivities and Fisher information, so that a model is penalized or rejected when different parameters produce nearly indistinguishable trajectory effects. The neural component remains inside the ODE vector field, but its width, depth, and parameterization are selected using predictive error together with the smallest Fisher-information eigenvalue, effective rank, and confidence intervals.

Formulas

$$\dot y(t)=f(y(t),\theta,t),\qquad \dot S(t)=f_y(y(t),\theta,t)S(t)+f_\theta(y(t),\theta,t),\qquad S(t_0)=\frac{\partial y_0}{\partial\theta}.$$

$$F(\theta)=\sum_{k=1}^{K}S(t_k)^T H_k^T\Sigma_k^{-1}H_kS(t_k),\qquad C_\theta\approx(F(\theta)+\lambda I)^{-1}.$$

$$\mathcal{J}_{arch}=L_{val}+\alpha\log\det(F+\lambda I)^{-1}+\beta p,\qquad \lambda_{\min}(F)\geq\tau,\quad \kappa(F)=\frac{\lambda_{\max}(F)}{\max(\lambda_{\min}(F),\lambda)}\leq\kappa_{max}.$$

Mathematical statement

Let the neural ODE be $\dot y(t)=f(y(t),\theta,t)$ with state $y(t)\in\mathbb{R}^{n}$ and trainable parameter vector $\theta=[\psi_M,\phi_{NN}]\in\mathbb{R}^{p}$. For each parameter, integrate the forward sensitivity matrix $S(t)=\partial y(t)/\partial\theta\in\mathbb{R}^{n\times p}$ using $\dot S(t)=f_y(y(t),\theta,t)S(t)+f_\theta(y(t),\theta,t)$, where $f_y=\partial f/\partial y$ and $f_\theta=\partial f/\partial\theta$. If observations are $x_k=H_k y(t_k)+\epsilon_k$ with observation matrix $H_k$ and noise covariance $\Sigma_k$, the local Fisher information matrix is $F(\theta)=\sum_k S_k^T H_k^T\Sigma_k^{-1}H_k S_k$, where $S_k=S(t_k)$. Small eigenvalues of $F$ indicate weakly identifiable parameter directions; under a locally Gaussian approximation, parameter covariance is $C_\theta\approx(F+\lambda I)^{-1}$, with ridge $\lambda>0$ for numerical stability. Select an architecture by minimizing $L_{val}+\alpha\log\det(F+\lambda I)^{-1}+\beta p$, subject to a minimum-eigenvalue or condition-number constraint.

Implementation notes

1. Exact integration point: implement the sensitivity calculation in a neural ODE solver used for hybrid dynamics, latent-state models, or continuous-time sequence prediction. Start with a vector field $f(y,\theta,t)$ containing both mechanistic parameters and neural weights. During candidate architecture training, augment the ODE state with the flattened sensitivity matrix $S$ and integrate the sensitivity equation together with $y$; alternatively use automatic-differentiation Jacobian-vector products if the full $n\times p$ matrix is too large. At observation times, compute residuals and accumulate the Fisher matrix using the known observation mask $H_k$ and an estimated diagonal noise covariance $\Sigma_k$. 2. Pseudocode: for each candidate width and depth, initialize $\theta$; repeatedly solve $(y,S)$ with $\dot y=f(y,\theta,t)$ and $\dot S=f_yS+f_\theta$; compute trajectory loss and optimize $\theta$; after convergence form $F=\sum_kS_k^TH_k^T\Sigma_k^{-1}H_kS_k$; reject the candidate if $\lambda_{min}(F)<\tau$ or $\kappa(F)>\kappa_{max}$; otherwise rank it by $L_{val}+\alpha\log\det(F+\lambda I)^{-1}+\beta p$. 3. Computed from the mechanism: sensitivity dynamics, Fisher matrix, covariance approximation, and architecture score. Estimated empirically: noise variances, thresholds $\tau$ and $\kappa_{max}$, and weights $\alpha,\beta$. 4. First cheap experiment: generate noisy trajectories from a two-dimensional damped oscillator or Lotka-Volterra system, train ordinary NODEs with widths 8, 16, 32, and 64, and compare them with the Fisher-ranked procedure using identical train and validation data. Evaluate in-distribution fit and extrapolation beyond the observed interval. The quantitative prediction is that wider models will eventually show a sharp collapse in $\lambda_{min}(F)$ and a rapid increase in $\kappa(F)$ despite similar validation loss; the selected model should occur before this identifiability transition, reduce median parameter confidence widths, and improve extrapolation. Verify that the measured uncertainty ranking is stable across noise seeds and that the selected model satisfies the target condition number within 20% across independently simulated trajectories.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a sensitivity-augmented ODE/Fisher MVP and a small width-selection experiment. Sensitivity finite differences matched the integrated sensitivity equation with maximum error 2.7e-15; the smallest Fisher eigenvalue scaled as delta^1.92 versus the predicted delta^2, covariance trace scaled as noise sigma^2, and exact duplicated parameters caused rank collapse. Fisher ranking selected width 4 instead of validation-only width 16, with worsening condition numbers for wider models, but this architecture result is preliminary.

Agent confidence: 8/10

Baseline: Validation-only selection chose width 16 with validation MSE 0.001084; widths 4, 8, and 32 gave 0.002143, 0.001564, and 0.001239.
Idea: Fisher score selected width 4 with score 1.440; the controlled redundancy slope was 1.922 versus predicted 2.0, covariance-noise slope was 2.000 versus predicted 2.0, and exact duplicate rank was 1. Width 32 had score 26.10 and condition number 3.10e10 versus 8.22e9 for width 4.

Limitations: The architecture comparison used a fixed-step Euler NODE and one seed, with a low-rank trajectory-observation Jacobian. Continuous adaptive solvers, true extrapolation improvement, confidence-interval calibration, multiple noise seeds, and independent-trajectory condition-number stability were not tested.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.13044) Sep 2, 2026

Significantly WORSE than the tuned baseline (+20.7%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0008
Idea mean: 0.0009
Effect (Δ): 0.0002 (+20.7%; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±13.8%
Mechanism: Confirmed ✓
Practical effect: Hurts
Baseline tuning: swept over 3 configs

Limitations:

Only the built-in dynamics track was tested; no architecture-width selection, continuous adaptive neural-ODE solver, confidence-interval calibration, extrapolation horizon study, or independent trajectory condition-number stability analysis was performed. The implemented Fisher term uses an output-sensitivity Gram logdet probe approximation rather than a full parameter-by-parameter continuous-time sensitivity ODE.

How to run: python3 bench_experiment.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Failed ✗

Agent confidence: 10/10

Built Fisher-identifiability regularization for the matched dynamics track using output Jacobian Gram log-determinant, with canonical rnn_small models and shared learning-rate search space. The mechanism signature was confirmed: observed Fisher log-volume improved by 3.98 while effective rank stayed equal, but test MSE worsened significantly from 0.0007802 to 0.0009419 (paired delta +0.0001617, permutation p=0.0081). Therefore this idea did not produce a benchmark win.

Baseline: Best lr=0.006; 8-seed test MSE 0.0007801908 ± 0.0001547640.
Idea: Best matched Fisher setting lr=0.006, alpha=0.004; 8-seed test MSE 0.0009418904 ± 0.0001986655; paired delta +0.0001616996, p=0.0081, 0/8 wins.
Mechanism signature: Confirmed: trained-model Fisher logdet mean improved from -21.0056 baseline to -17.0287 idea (gain +3.9768); effective rank was 8.0 for both.

Benchmark result