✗ Failed on benchmark 2026

Sensitivity-Conditioned Neural ODE Pruning

Implementation & benchmark of arXiv:2608.13044 — Identifiability-aware neural ordinary differential equations for parsimonious and reliable dynamic modelling

Usefulness7/10

Difficulty5/10

Novelty8/10

Architecture Stability Generalization Speedup Ssm Rnn Pruning Inference speedup Regularization Dynamical systems Statistics Linear algebra Optimization

Source paper: Identifiability-aware neural ordinary differential equations for parsimonious and reliable dynamic modelling arXiv:2608.13044 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Use trajectory sensitivities to remove neural units or parameter groups whose effects are redundant over the available data support. A parameter group is pruned when its Fisher contribution is small or its sensitivity is nearly collinear with other groups, producing a compact neural ODE without relying only on parameter magnitude.

Formulas

$$J_g=\begin{bmatrix}H_1S(t_1)[:,g]\\ \vdots\\ H_KS(t_K)[:,g]\end{bmatrix},\qquad I_g=J_g^TWJ_g.$$

$$r_g=\left\|(I-P_{-g})W^{1/2}J_g\right\|_F^2,\qquad P_{-g}=A_{-g}(A_{-g}^TA_{-g}+\epsilon I)^{-1}A_{-g}^T,$$

$$\theta_g\ \text{is pruned if}\quad \operatorname{tr}(I_g)<\delta_I\ \text{or}\ r_g<\delta_R.$$

Mathematical statement

Partition the parameter vector into groups $\theta=(\theta_1,\ldots,\theta_G)$, where a group may be one hidden unit, one neuron's incoming weights, or one mechanistic coefficient. Let $J_g$ be the stacked observed-output sensitivity for group $g$, with $J_g$ formed from $H_kS(t_k)[:,g]$ over all observation times. The group information contribution is $I_g=J_g^TWJ_g$, where $W$ is the block-diagonal inverse observation-noise covariance. A group is redundant when its sensitivity is small, $\operatorname{tr}(I_g)$ is low, or its sensitivity lies in the span of other groups, measured by $r_g=\|(I-P_{-g})W^{1/2}J_g\|_F^2$, where $P_{-g}$ projects onto the weighted sensitivity span of all remaining groups. Iterative pruning keeps groups with sufficient incremental sensitivity rank and retrains after each pruning round.

Implementation notes

1. Exact integration point: apply this after ordinary neural ODE training, before final retraining, to hidden units or parameter blocks in the neural component of the vector field. Do not prune using raw parameter magnitude; collect each group's sensitivity over the actual training trajectories and observed time points. 2. Pseudocode: train a candidate NODE; solve the augmented state-sensitivity system or obtain $S_k$ with automatic differentiation; construct weighted sensitivity columns $A_g=W^{1/2}J_g$; compute each group's $\operatorname{tr}(I_g)$ and residual $r_g$ against the span of all other groups; remove the lowest-scoring group if either score is below threshold; retrain the reduced model; repeat until validation error increases or every remaining group exceeds the threshold. Use QR factorization or randomized low-rank linear algebra rather than explicitly forming a dense Fisher matrix for large networks. 3. Taken from the mechanism: sensitivity-based information and redundancy tests. Estimated empirically: pruning thresholds $\delta_I,\delta_R$, observation-noise weighting $W$, and the stopping criterion. 4. First cheap experiment: use a width-64 neural ODE to model noisy Lotka-Volterra or damped-oscillator trajectories, then compare magnitude pruning, random pruning, and sensitivity-conditioned pruning at equal parameter budgets of 50%, 25%, and 10% of the original size. Measure one-step error, long-horizon trajectory error, solver function evaluations, parameter count, and the Fisher condition number. The prediction is that sensitivity pruning removes parameters with low incremental Fisher rank while preserving validation error, whereas magnitude pruning sometimes removes dynamically important but small-weight parameters. A successful run should show approximately unchanged short-horizon error, lower long-horizon error than the unpruned overparameterized model, and solver cost decreasing with retained units; after retraining, the smallest nonzero Fisher eigenvalue should not decrease by more than 20%.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a compact neural-ODE unit-pruning experiment with weighted Fisher traces and leave-one-group sensitivity residuals, plus reproducible numerical verification. The mechanism checks passed: observed exponents were 2.000 for both information and residual amplitude scaling, -2.000 for noise scaling, and the collinearity residual matched C·sin(angle)^2 with ratio CV 1.0e-15. At 8/16 retained units, sensitivity pruning reached MSE 1.27e-6 and terminal error 0.00221 versus magnitude pruning MSE 2.31e-6 and terminal error 0.00297, but random pruning was better at MSE 3.94e-7 and terminal error 0.00131; therefore the mathematical mechanism manifested, but a practical accuracy win over random pruning was not demonstrated.

Agent confidence: 8/10

Baseline: Magnitude: 24 parameters, MSE 2.305e-6, terminal error 0.002974; random: 24 parameters, MSE 3.943e-7, terminal error 0.001313
Idea: Sensitivity: 24 parameters, MSE 1.266e-6, terminal error 0.002210; information/residual/noise observed scaling exponents 2.000/2.000/-2.000; angle-law ratio CV 1.05e-15

Limitations: Only a scalar damped nonlinear ODE and one 50% retention budget were tested; the requested 25% and 10% budgets, Lotka-Volterra trajectories, validation stopping, solver function-evaluation speed, Fisher condition numbers, and multiple random seeds were not evaluated. The sensitivity estimate uses finite differences through a fixed-step RK4 integrator rather than an augmented continuous adjoint sensitivity solve.

How to run: python3 sensitivity_pruning_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.13044) Sep 2, 2026

Significantly WORSE than the tuned baseline (n/a, p=0.0081, wins 0 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0
Idea mean: 0
Effect (Δ): 0.0009 (n/a; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: n/a
Mechanism: Not confirmed ✗
Practical effect: Hurts
Baseline tuning: swept over 3 configs

Limitations:

Only the built-in dynamics track, one 50% retention level, 15 training epochs, and GRU head/hidden-unit groups were tested; 25% and 10% retention, iterative multi-round pruning, Fisher condition numbers, solver function evaluations, and continuous neural-ODE integration were not tested. The benchmark's rnn_small is a recurrent pendulum predictor rather than a continuous-time NODE.

How to run: python3 bench_sensitivity_pruning.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Failed ✗

Agent confidence: 9/10

Built and ran sensitivity-conditioned hidden-unit pruning on the structurally matched dynamics/rnn_small benchmark using the canonical training path, 8 paired seeds, and a parity-compliant three-value learning-rate sweep. The idea's best MSE was 0.00127219 versus the tuned magnitude baseline's 0.00041839 (paired delta +0.00085380, permutation p=0.0081), making the idea significantly worse. The trained-model information-to-ablation correlation averaged below the confirmation threshold, so the proposed mechanism was not quantitatively confirmed at neural-network scale.

Baseline: dynamics/rnn_small, magnitude pruning, best lr=0.006, 32/64 hidden units retained, full 8-seed mean MSE=0.00041839 (exact value recorded in bench_report.json)
Idea: dynamics/rnn_small, sensitivity information plus leave-one-group residual pruning, best lr=0.006, 32/64 hidden units retained, full 8-seed mean MSE=0.0012721884; paired delta=+0.0008537985, p=0.0081, 0/8 wins
Mechanism signature: map[confirmed:false interpretation:sensitivity information was only moderately correlated with observed hidden-unit ablation impact and did not support the claimed selection benefit mean_importance_ablation_correlation:0.4416]

Benchmark result