Residual-to-Symbolic Neural Pruning
Implementation & benchmark of arXiv:2608.00466 — SPIRAL-PO: Symbolic Identification of Partially Observed Nonlinear Dynamics with Application to Rotating Machinery
Source paper: SPIRAL-PO: Symbolic Identification of Partially Observed Nonlinear Dynamics with Application to Rotating Machinery arXiv:2608.00466 ⓘ · analyzed Aug 31, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Use a two-stage residual augmentation loop: first let a residual network explain model mismatch, then project its learned vector field onto a physically constrained candidate library and replace the flexible residual with the accepted sparse terms. This turns an unconstrained neural correction into a low-complexity dynamical law that is easier to roll out over long horizons and can expose unsupported hidden-state explanations.
Formulas
Mathematical statement
Given observed state $z_o$, input $u$, and base vector field $F_o^{obs}$, define the residual target $r(t)=\dot z_o(t)-F_o^{obs}(z_o(t),u(t),p)$. Fit a residual network $R_\theta(z_o,u)$, then project its outputs onto library features $\Phi(z_o,u)$ by solving $\min_B\|R_\theta-B\Phi\|_2^2+\lambda\|B\|_1$. For each candidate coefficient $b_j$, use an estimated standard error $\mathrm{se}(b_j)$ and admit the term only if $|b_j|/\mathrm{se}(b_j)>\tau$ after accounting for the other selected terms. The final model is $\dot z_o=F_o^{obs}+B_{\mathrm{sel}}\Phi$, optionally coupled to a latent neural state estimator. The mechanism is a bias-variance and extrapolation trade: the neural residual supplies discovery flexibility, while the sparse projected field limits rollout amplification.
Implementation notes
Integrate this into the transition function of a neural ODE, SSM, or recurrent world model. Construct $F_o^{obs}$ from known equations when available; otherwise use a small baseline network frozen for one projection cycle. Define a library containing only dimensionally valid and symmetry-compatible terms, such as $y$, $y^3$, $u$, $yu$, velocity-like finite differences, and cross-coordinate products. First train an MLP residual $R_\theta$ against the one-step derivative residual. Then freeze the MLP, evaluate it on a trajectory buffer, and fit $B$ by elastic net or sequential thresholded least squares. Refit the residual network after replacing its output with $B_{sel}\Phi$, repeating for two to five cycles. For uncertainty, bootstrap trajectories or use the regression covariance $\hat\sigma^2(X^\top X+\lambda I)^{-1}$, where $X$ is the library design matrix; the paper's statistical-gating mechanism is implemented through coefficient $t$-statistics, while the covariance estimate is empirical. The first experiment should use a noisy nonlinear oscillator with a known cubic restoring force and an unobserved auxiliary mode. Compare an unconstrained residual neural ODE, sparse regression alone, and the alternating method on one-step and 100-step rollout error. The falsifiable prediction is that training error will initially favor the unconstrained residual, but after projection the selected model will have lower long-horizon error and lower seed-to-seed variance; accepted terms should remain significant across bootstrap runs, while false library terms should be removed when their $|t|$ statistic is below the chosen threshold.
Verification
Stage 1 · Toy mechanism gate: Failed ✗
Stage 2 · Mechanism transferred to benchmark: Not tested
Stage 2 · Practical benchmark result: Not run
Methodology: Toy-system gate first; the benchmark stage runs only after a pass. How verification works
Stage 1 — Mechanism check agent confidence 8/10
Built a reproducible noisy nonlinear-oscillator experiment with an MLP residual, L1 library projection, coefficient t-statistic gating, and sparse-only/neural rollout baselines. The statistical mechanism showed the expected qualitative behavior: signal t-statistics increased with sample size (observed exponent 0.606 versus predicted 0.5) and null acceptance was low (0.0178 versus predicted 0.0124), but the endpoint ratio was 5.34 versus predicted 4.0. The projected symbolic model improved acceleration rollout RMSE over the neural residual (0.05294 versus 0.06442), yet it was identical to sparse regression alone and admitted spurious v and v3 terms, so the claimed residual-to-symbolic advantage and reliable hidden-state term rejection were not demonstrated.
- Agent confidence
- 8/10
- Baseline
- Unconstrained neural residual: training RMSE 0.05683; 100-step acceleration rollout RMSE 0.06442. Sparse-only rollout RMSE 0.05294.
- Idea
- Residual-to-symbolic rollout RMSE 0.05294, selected x3, u, v, v3; t-statistics for v and v3 were 10.13 and 11.93. Mechanism sweep: t scaling exponent 0.606 vs predicted 0.5, 640/40 t ratio 5.34 vs predicted 4.0, null acceptance 0.0178 vs predicted 0.0124.
Limitations: The rollout metric is acceleration error rather than full observed-state trajectory error, only one fixed seed was used for the model comparison, no multi-cycle refitting or bootstrap uncertainty was tested, and the latent auxiliary mode was not estimated. The sparse projection was effectively equivalent to direct sparse regression because the final accepted coefficients were refit against the measured residual.
How to run: python3 experiment.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Artifacts
- experiment.py 7.1 KB View Raw
- report.md 1.9 KB View
- results.json 1.9 KB View Raw