SynthCore
✗ Mechanism failed Re-invented 2026

Progressive rollout-consistency training

Implementation & benchmark of arXiv:2608.22112 — Symbolic Neural ODEs: Learning interpretable models from time-series data

Usefulness8/10
Difficulty5/10
Novelty5/10
Dynamics StabilityAccuracySample efficiencyWorld modelTraining dynamicsNeural odeDynamical systemsNumerical analysisOptimization

Source paper: Symbolic Neural ODEs: Learning interpretable models from time-series data arXiv:2608.22112 ⓘ · analyzed Aug 29, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Train a learned vector field through the same finite-step integrator used for forecasting, rather than fitting only instantaneous derivatives or one-step transitions. Increase the rollout horizon during optimization and combine multi-step error with sparsity, forcing the model to remain self-consistent under repeated composition and reducing exposure bias.

Formulas

$$\frac{d\bm{x}}{dt}=\bm{f}(\bm{x}),\qquad \bm{x}\in\mathbb{R}^{n},\quad \bm{f}:\mathbb{R}^{n}\to\mathbb{R}^{n}.$$
$$\tilde{\bm{x}}[h+1]=\tilde{\bm{x}}[h]+\frac{1}{6}(\bm{k}_{1}+2\bm{k}_{2}+2\bm{k}_{3}+\bm{k}_{4})\equiv\tilde{\bm{S}}(\tilde{\bm{x}}[h]),$$
$$\bm{k}_{1}=\Delta t\tilde{\bm{f}}(\tilde{\bm{x}}[h]),\quad \bm{k}_{2}=\Delta t\tilde{\bm{f}}(\tilde{\bm{x}}[h]+\bm{k}_{1}/2),\quad \bm{k}_{3}=\Delta t\tilde{\bm{f}}(\tilde{\bm{x}}[h]+\bm{k}_{2}/2),\quad \bm{k}_{4}=\Delta t\tilde{\bm{f}}(\tilde{\bm{x}}[h]+\bm{k}_{3}).$$
$$\mathcal{L}_{H}(\theta)=\frac{1}{H}\sum_{j=1}^{H}\left\|\tilde{\bm{x}}[j]-\bm{x}[j]\right\|_{1}+\lambda\|\theta\|_{1},\qquad \tilde{\bm{x}}[0]=\bm{x}[0].$$

Mathematical statement

The paper models an autonomous system as $d\bm{x}/dt=\bm{f}(\bm{x})$, where $\bm{x}\in\mathbb{R}^{n}$ is the state and $\bm{f}:\mathbb{R}^{n}\to\mathbb{R}^{n}$ is the vector field. Its explicit fourth-order Runge–Kutta step defines a discrete map $\tilde{\bm{S}}$ by $\tilde{\bm{x}}[h+1]=\tilde{\bm{x}}[h]+(\bm{k}_{1}+2\bm{k}_{2}+2\bm{k}_{3}+\bm{k}_{4})/6$, with $\bm{k}_{1}=\Delta t\tilde{\bm{f}}(\tilde{\bm{x}}[h])$, $\bm{k}_{2}=\Delta t\tilde{\bm{f}}(\tilde{\bm{x}}[h]+\bm{k}_{1}/2)$, $\bm{k}_{3}=\Delta t\tilde{\bm{f}}(\tilde{\bm{x}}[h]+\bm{k}_{2}/2)$, and $\bm{k}_{4}=\Delta t\tilde{\bm{f}}(\tilde{\bm{x}}[h]+\bm{k}_{3})$. Here $h$ is the discrete time index, $\Delta t$ is the sampling interval, $\tilde{\bm{f}}$ is the learned vector field, and $\tilde{\bm{S}}$ is the learned one-step transition. Use the adapted objective $\mathcal{L}_{H}=\frac{1}{H}\sum_{j=1}^{H}\|\tilde{\bm{x}}[j]-\bm{x}[j]\|_{1}+\lambda\|\theta\|_{1}$, where $H$ is the rollout horizon, $\theta$ contains vector-field parameters, and $\lambda$ controls sparsity. The horizon schedule $H=1\rightarrow H_{\max}$ first provides stable gradients and then penalizes errors amplified by repeated composition.

Implementation notes

Integrate this at the training loss of a learned dynamics or world-model module, replacing teacher-forced one-step prediction with differentiable RK4 rollouts. Inputs are a batch of trajectories $x_{b,0:H_{\max}}$ and a vector-field network $f_{\theta}$. For each curriculum phase, choose horizon $H$, set $\hat{x}_{b,0}=x_{b,0}$, and execute the following loop: compute $k_1=dt f_\theta(\hat{x}_{b,j})$; compute $k_2=dt f_\theta(\hat{x}_{b,j}+k_1/2)$; compute $k_3=dt f_\theta(\hat{x}_{b,j}+k_2/2)$; compute $k_4=dt f_\theta(\hat{x}_{b,j}+k_3)$; then set $\hat{x}_{b,j+1}=\hat{x}_{b,j}+(k_1+2k_2+2k_3+k_4)/6$. Accumulate the mean absolute error against $x_{b,j+1}$ and add $\lambda\|\theta\|_1$. Start with $H=1$ for several epochs, then increase geometrically, for example $1,2,4,8,16$, while optionally normalizing coordinates and clipping gradients. The RK4 formula and loss are taken from the paper; the schedule, gradient clipping, and normalization are engineering choices. Measure rollout error at horizons longer than training, maximum state norm, gradient explosions, and long-time means and variances. The first cheap test is a 2D pendulum, Van der Pol, and Lorenz system with 1% additive noise, comparing one-step MSE training, fixed long-horizon training, and the curriculum at equal vector-field evaluations. Success means lower 32- or 64-step error, fewer divergent trajectories, and improved recovery of long-time statistics at equal training FLOPs.

Verification

Mechanism failed

Stage 1 · Toy mechanism gate: Failed ✗

Stage 2 · Mechanism transferred to benchmark: Not tested

Stage 2 · Practical benchmark result: Not run

Methodology: Toy-system gate first; the benchmark stage runs only after a pass. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built differentiable RK4 rollout training with L1 regularization, progressive horizons, gradient clipping, synthetic noisy nonlinear oscillator data, and long-horizon evaluation. The corrected numerical sanity check confirmed fourth-order RK4 scaling (error ratios 17.09 and 16.59 under timestep halving), but at approximately equalized training evaluations the one-step baseline was better at every horizon: 64-step MAE 0.04368 versus 0.07336 for progressive training. No trajectories diverged in either method, so the promised forecasting/stability win was not observed in this MVP.

Agent confidence
9/10
Baseline
MAE at horizons 8/16/32/64: 0.01121/0.01994/0.03112/0.04368; max norm 1.586; divergence rate 0.0
Idea
MAE at horizons 8/16/32/64: 0.01759/0.03150/0.05122/0.07336; max norm 1.591; divergence rate 0.0; RK4 halving ratios 17.09 and 16.59

Limitations: Only one seed and one stable 2D synthetic oscillator were tested; no pendulum, Van der Pol, Lorenz, multiple noise levels, hyperparameter tuning, wall-clock/FLOP profiling, sparse-model recovery, or statistical significance analysis was performed.

How to run: python3 experiment.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Artifacts

Implementation overview ⬇ Download all as ZIP 3 files · code, reports and structured results

Related ideas