# Pseudo-Arclength Continuation for Neural ODE Attractors

- ID: 402
- Canonical URL: https://synthcore.org/idea/402/pseudo-arclength-continuation-for-neural-ode-attractors
- API JSON: https://synthcore.org/api/idea/402.json
- API Markdown: https://synthcore.org/api/idea/402.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.03498](https://arxiv.org/abs/2607.03498)
- Category: dynamics
- Solves: stability, sample-efficiency, generalization
- ML areas: world-model, training-dynamics, initialization
- Math tags: dynamical-systems, bifurcations, numerical-analysis, harmonic-analysis
- Ratings: usefulness 7/10; difficulty 7/10; novelty 8/10

## Idea description

Use the paper's parameterized invariant-torus residual and pseudo-arclength Newton correction to train a neural ODE across a continuous family of latent dynamical regimes. The continuation constraint allows the solver to pass through saddle-node folds, where stepping a physical control parameter alone would fail or jump to a different branch.

## Mathematical statement

For an autonomous neural ODE \(\dot{x}=f_\mu(x)\) with state \(x\in\mathbb{R}^n\) and system parameters \(\mu\in\mathbb{R}^p\), represent a \(d\)-frequency invariant torus by a periodic parameterization \(K:\mathbb{T}^d\to\mathbb{R}^n\), where \(\theta\in\mathbb{T}^d\) and the prescribed frequency vector is \(\omega\in\mathbb{R}^d\). The invariance equation is \(F(K,\mu)(\theta)=f_\mu(K(\theta))-DK(\theta)\omega=0\). A Newton step solves \(DF(K,\mu)[\Delta K,\Delta\mu]=-F(K,\mu)\), with \(DF[\Delta K,\Delta\mu]=D_xf_\mu(K)\Delta K+D_\mu f_\mu(K)\Delta\mu-D(\Delta K)\omega\). Along a solution branch \(z=(K,\mu)\), pseudo-arclength continuation adds \(\langle z-z_\mathrm{pred},t_\mathrm{pred}\rangle=0\), where \(t_\mathrm{pred}\) is the normalized tangent from the previous two converged solutions and \(z_\mathrm{pred}=z_\mathrm{old}+\Delta s\,t_\mathrm{pred}\). This extra scalar equation permits continuation through a fold where \(\partial\mu/\partial s=0\). In a Fourier discretization, \(K(\theta)=\sum_{k\in\mathcal{K}}\hat K_k e^{i k\cdot\theta}\) and the directional derivative is \(\widehat{DK\omega}_k=i(k\cdot\omega)\hat K_k\); the potentially difficult denominators are the small divisors \(k\cdot\omega\).

## Key formulas

- $$F(K,\mu)(\theta)=f_\mu(K(\theta))-DK(\theta)\,\omega=0,$$
- $$D F(K,\mu)[\Delta K,\Delta\mu]=D_xf_\mu(K)\Delta K+D_\mu f_\mu(K)\Delta\mu-D(\Delta K)\omega,$$
- $$\begin{bmatrix}DF(K,\mu)\\ t_{\mathrm{pred}}^{\mathsf T}\end{bmatrix}\begin{bmatrix}\Delta K\\ \Delta\mu\end{bmatrix}=-\begin{bmatrix}F(K,\mu)\\ \langle z-z_{\mathrm{pred}},t_{\mathrm{pred}}\rangle\end{bmatrix},\qquad z=(K,\mu),$$
- $$K(\theta)=\sum_{k\in\mathcal K}\hat K_k e^{i k\cdot\theta},\qquad \widehat{DK\omega}_k=i(k\cdot\omega)\hat K_k.$$

## Implementation notes

Integrate this at the training layer of a small neural ODE or latent world model, not inside ordinary minibatch backpropagation. Let the neural vector field be \(f_\mu(x;\phi)\), with trainable weights \(\phi\) and a low-dimensional continuation parameter \(\mu\) such as forcing amplitude, damping, or a learned scalar gate. Represent the candidate torus by Fourier coefficients \(\hat K_k\in\mathbb{C}^n\) for \(|k_j|\leq M\), and evaluate \(K\) and \(DK\omega\) on an \(L^d\) torus grid using FFTs. At each continuation point: (1) initialize \(K,\mu\) from the previous converged point; (2) compute the residual \(r=f_\mu(K;\phi)-DK\omega\); (3) estimate Jacobian-vector products of \(r\) with automatic differentiation; (4) solve the augmented linear system consisting of the Newton residual equation and the pseudo-arclength row with GMRES; (5) apply damped updates to \(K\) and \(\mu\) until the normalized residual \(\|r\|_2/\|f_\mu(K)\|_2\) is below tolerance; (6) optionally update \(\phi\) by minimizing the same residual over several continuation points. The tangent is \(t=(z_j-z_{j-1})/\|z_j-z_{j-1}\|\), with sign chosen to agree with the preceding tangent. The paper supplies the continuation and Newton structure; estimate Jacobian-vector products empirically through autodiff, while all Fourier derivatives use the exact multiplier \(i(k\cdot\omega)\). Start with a two-dimensional latent neural ODE, \(d=2\), \(M=4\), and an irrational frequency vector such as \(\omega=(1,(\sqrt5-1)/2)\). Compare ordinary parameter stepping against pseudo-arclength on a forced oscillator with a known saddle-node of invariant tori. Measure the fraction of continuation steps converged, branch-jump frequency, residual, and downstream forecasting error. Success is a continuous branch through the fold with fewer failed solves and lower forecasting error at equal numbers of trained parameter values.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: inconclusive
- Verdict: Built and numerically verified a Fourier pseudo-arclength continuation toy for the saddle-node branch x²+μ−1=0. The mechanism manifested: the Fourier multiplier identity had max error 3.64e-14, the observed conditioning log-log slope was 1.0000000000000007 versus predicted 1, and pseudo-arclength converged all 24 steps for every tested step size, crossing the fold at ds=0.2 and reaching μ=-1.78. Ordinary parameter stepping became iteration-limited near and after the fold, converging only 83.6–85.7% of requested points.

### Mechanism check

- Verdict: Built and numerically verified a Fourier pseudo-arclength continuation toy for the saddle-node branch x²+μ−1=0. The mechanism manifested: the Fourier multiplier identity had max error 3.64e-14, the observed conditioning log-log slope was 1.0000000000000007 versus predicted 1, and pseudo-arclength converged all 24 steps for every tested step size, crossing the fold at ds=0.2 and reaching μ=-1.78. Ordinary parameter stepping became iteration-limited near and after the fold, converging only 83.6–85.7% of requested points.
- Confidence: 8/10
- Limitations: This is a low-dimensional scalar saddle-node toy rather than the proposed 2D neural ODE with a learned vector field, d=2 torus residual, autodiff JVPs, GMRES, training, or forecasting evaluation. The implementation uses a dense 2x2 solve and does not test small-divisor effects, noisy residuals, branch-jump frequency, FLOPs, or downstream ML generalization.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.86075
- Paired wins: 3/8
- Benchmark verdict: no significant win

## Artifacts

- [bench_report.json](https://synthcore.org/code/205/bench_report.json)
- [continuation_results.json](https://synthcore.org/code/205/continuation_results.json)
- [continuation_summary.json](https://synthcore.org/code/205/continuation_summary.json)
- [continuation_toy.py](https://synthcore.org/code/205/continuation_toy.py)
- [report.md](https://synthcore.org/code/205/report.md)
- [report_bench_2026-08-30T201220.md](https://synthcore.org/code/205/report_bench_2026-08-30T201220.md)
- [run_stage2.py](https://synthcore.org/code/205/run_stage2.py)
- [stage2_continuation.py](https://synthcore.org/code/205/stage2_continuation.py)
- [summarize_results.py](https://synthcore.org/code/205/summarize_results.py)
- [Download all files as ZIP](https://synthcore.org/download/205)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.