✗ Failed on benchmark 2026

Pseudo-Arclength Continuation for Neural ODE Attractors

Implementation & benchmark of arXiv:2607.03498 — Numerical Computation of Quasiperiodic Reducible Saddle-Node Bifurcations: a Parameterization Method Approach

Usefulness7/10

Difficulty7/10

Novelty8/10

Dynamics Stability Sample efficiency Generalization World model Training dynamics Initialization Dynamical systems Bifurcations Numerical analysis Harmonic analysis

Source paper: Numerical Computation of Quasiperiodic Reducible Saddle-Node Bifurcations: a Parameterization Method Approach arXiv:2607.03498 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Use the paper's parameterized invariant-torus residual and pseudo-arclength Newton correction to train a neural ODE across a continuous family of latent dynamical regimes. The continuation constraint allows the solver to pass through saddle-node folds, where stepping a physical control parameter alone would fail or jump to a different branch.

Formulas

$$F(K,\mu)(\theta)=f_\mu(K(\theta))-DK(\theta)\,\omega=0,$$

$$D F(K,\mu)[\Delta K,\Delta\mu]=D_xf_\mu(K)\Delta K+D_\mu f_\mu(K)\Delta\mu-D(\Delta K)\omega,$$

$$\begin{bmatrix}DF(K,\mu)\\ t_{\mathrm{pred}}^{\mathsf T}\end{bmatrix}\begin{bmatrix}\Delta K\\ \Delta\mu\end{bmatrix}=-\begin{bmatrix}F(K,\mu)\\ \langle z-z_{\mathrm{pred}},t_{\mathrm{pred}}\rangle\end{bmatrix},\qquad z=(K,\mu),$$

$$K(\theta)=\sum_{k\in\mathcal K}\hat K_k e^{i k\cdot\theta},\qquad \widehat{DK\omega}_k=i(k\cdot\omega)\hat K_k.$$

Mathematical statement

For an autonomous neural ODE $\dot{x}=f_\mu(x)$ with state $x\in\mathbb{R}^n$ and system parameters $\mu\in\mathbb{R}^p$, represent a $d$-frequency invariant torus by a periodic parameterization $K:\mathbb{T}^d\to\mathbb{R}^n$, where $\theta\in\mathbb{T}^d$ and the prescribed frequency vector is $\omega\in\mathbb{R}^d$. The invariance equation is $F(K,\mu)(\theta)=f_\mu(K(\theta))-DK(\theta)\omega=0$. A Newton step solves $DF(K,\mu)[\Delta K,\Delta\mu]=-F(K,\mu)$, with $DF[\Delta K,\Delta\mu]=D_xf_\mu(K)\Delta K+D_\mu f_\mu(K)\Delta\mu-D(\Delta K)\omega$. Along a solution branch $z=(K,\mu)$, pseudo-arclength continuation adds $\langle z-z_\mathrm{pred},t_\mathrm{pred}\rangle=0$, where $t_\mathrm{pred}$ is the normalized tangent from the previous two converged solutions and $z_\mathrm{pred}=z_\mathrm{old}+\Delta s\,t_\mathrm{pred}$. This extra scalar equation permits continuation through a fold where $\partial\mu/\partial s=0$. In a Fourier discretization, $K(\theta)=\sum_{k\in\mathcal{K}}\hat K_k e^{i k\cdot\theta}$ and the directional derivative is $\widehat{DK\omega}_k=i(k\cdot\omega)\hat K_k$; the potentially difficult denominators are the small divisors $k\cdot\omega$.

Implementation notes

Integrate this at the training layer of a small neural ODE or latent world model, not inside ordinary minibatch backpropagation. Let the neural vector field be $f_\mu(x;\phi)$, with trainable weights $\phi$ and a low-dimensional continuation parameter $\mu$ such as forcing amplitude, damping, or a learned scalar gate. Represent the candidate torus by Fourier coefficients $\hat K_k\in\mathbb{C}^n$ for $|k_j|\leq M$, and evaluate $K$ and $DK\omega$ on an $L^d$ torus grid using FFTs. At each continuation point: (1) initialize $K,\mu$ from the previous converged point; (2) compute the residual $r=f_\mu(K;\phi)-DK\omega$; (3) estimate Jacobian-vector products of $r$ with automatic differentiation; (4) solve the augmented linear system consisting of the Newton residual equation and the pseudo-arclength row with GMRES; (5) apply damped updates to $K$ and $\mu$ until the normalized residual $\|r\|_2/\|f_\mu(K)\|_2$ is below tolerance; (6) optionally update $\phi$ by minimizing the same residual over several continuation points. The tangent is $t=(z_j-z_{j-1})/\|z_j-z_{j-1}\|$, with sign chosen to agree with the preceding tangent. The paper supplies the continuation and Newton structure; estimate Jacobian-vector products empirically through autodiff, while all Fourier derivatives use the exact multiplier $i(k\cdot\omega)$. Start with a two-dimensional latent neural ODE, $d=2$, $M=4$, and an irrational frequency vector such as $\omega=(1,(\sqrt5-1)/2)$. Compare ordinary parameter stepping against pseudo-arclength on a forced oscillator with a known saddle-node of invariant tori. Measure the fraction of continuation steps converged, branch-jump frequency, residual, and downstream forecasting error. Success is a continuous branch through the fold with fewer failed solves and lower forecasting error at equal numbers of trained parameter values.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built and numerically verified a Fourier pseudo-arclength continuation toy for the saddle-node branch x²+μ−1=0. The mechanism manifested: the Fourier multiplier identity had max error 3.64e-14, the observed conditioning log-log slope was 1.0000000000000007 versus predicted 1, and pseudo-arclength converged all 24 steps for every tested step size, crossing the fold at ds=0.2 and reaching μ=-1.78. Ordinary parameter stepping became iteration-limited near and after the fold, converging only 83.6–85.7% of requested points.

Agent confidence: 8/10

Baseline: Physical-parameter Newton converged fractions 0.836, 0.840, 0.846, and 0.857 for parameter steps 0.02, 0.05, 0.10, and 0.20; it became iteration-limited near and beyond the fold.
Idea: Pseudo-arclength Newton converged 24/24 continuation steps for ds=0.02, 0.05, 0.10, 0.20, and 0.40. Fold crossing was observed for ds=0.20 and 0.40, with minimum μ=-1.781 and -6.724 respectively. Fourier derivative max error was 3.64e-14, and conditioning matched 2|x| exactly.

Limitations: This is a low-dimensional scalar saddle-node toy rather than the proposed 2D neural ODE with a learned vector field, d=2 torus residual, autodiff JVPs, GMRES, training, or forecasting evaluation. The implementation uses a dense 2x2 solve and does not test small-divisor effects, noisy residuals, branch-jump frequency, FLOPs, or downstream ML generalization.

How to run: python3 continuation_toy.py && /home/maxwelhelp/main/bin/python3 summarize_results.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.03498) Sep 2, 2026

No significant difference vs. the tuned baseline (+2.3%, p=0.8608, wins 3 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0022
Idea mean: 0.0023
Effect (Δ): 0.0001 (+2.3%; negative = idea better)
Wins: 3 / 8 paired seeds
p-value: 0.8608 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±32.2%
Mechanism: Not confirmed ✗
Practical effect: Inconclusive
Baseline tuning: swept over 3 configs

Limitations:

The intervention is a local discrete continuation-style control-curvature penalty rather than a full Fourier d=2 invariant-torus Newton/GMRES solver, because the built-in dynamics task provides flattened actuated-pendulum windows. The run used 12 epochs and 400/100 train/test examples; long-horizon forecasting, explicit torus branch jumps, GMRES, and FLOP comparisons were not available in the harness.

How to run: python3 run_stage2.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 30, 2026 latest Failed ✗

Agent confidence: 10/10

Implemented and evaluated a matched rnn_small intervention on the structurally appropriate dynamics track. Baseline MSE was 0.0022474244 versus 0.0023001978 for the idea, with paired delta +0.0000527733 and permutation p=0.86075; this is not a significant win. The trained-model mechanism signature also failed to confirm the predicted curvature reduction.

Baseline: Best lr=0.006; 8-seed MSE mean 0.0022474244324257597, std 0.0004174303543825742.
Idea: Best lr=0.006, lambda=0.1; 8-seed MSE mean 0.0023001977533567697, std 0.000824645192905565.

Benchmark result