✗ Failed on benchmark 2026

Periodic-Orbit Continuation for Recurrent Inference

Implementation & benchmark of arXiv:2608.21256 — Loadability Limits Under Periodic Load Forcing

Usefulness7/10

Difficulty7/10

Novelty7/10

Dynamics Stability Speedup Rnn Ssm Inference speedup Dynamical systems Bifurcations Linear algebra Numerical analysis

Source paper: Loadability Limits Under Periodic Load Forcing arXiv:2608.21256 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

For a recurrent or implicit neural model driven by periodic inputs, solve for a periodic hidden-state orbit and continue that orbit as input amplitude or frequency changes. This replaces repeated cold starts from zero and should preserve convergence near parameter ranges where cold starts fail.

Formulas

$$R(z,p)=\Phi_T(z;p)-z=0$$

$$(D_z\Phi_T-I)\frac{dz}{dp}=-D_p\Phi_T$$

$$\sigma_{\min}(I-M)\to 0,\qquad M=D_z\Phi_T$$

$$R(z,p)=0,\qquad c_z^{\top}(z-z_{\mathrm{pred}})+c_p(p-p_{\mathrm{pred}})-\Delta s=0$$

Mathematical statement

Let z_{k+1}=G_k(z_k;p) be a recurrent or implicit neural update with period T and forcing parameter p. Define the period residual R(z,p)=Phi_T(z;p)-z, where Phi_T is the T-step state map. A periodic state satisfies R(z,p)=0. Differentiation along a regular branch gives (D_z Phi_T-I) dz/dp = -D_p Phi_T. Continuation becomes ill-conditioned when D_z Phi_T-I is singular. Since M=D_z Phi_T, this is the same as a Floquet multiplier +1. The smallest singular value sigma_min(I-M) therefore measures proximity to the cyclic-fold boundary.

Implementation notes

Apply the method to a tanh RNN, an SSM, or a DEQ layer receiving a periodic input. The integration point is inference-time state initialization. At an easy amplitude p_0, solve R(z,p_0)=0 by ordinary fixed-point iteration and store the full period orbit. For the next parameter value, predict the orbit using a secant step from the last two converged solutions, then correct it with Newton or quasi-Newton iterations on the period residual together with the pseudo-arclength constraint. Compute JVPs and VJPs through the unrolled period map, and estimate sigma_min(I-M) with randomized SVD or Lanczos rather than forming M explicitly. The paper contributes the fixed-point residual, continuation equation, and singularity criterion; the neural Jacobians and orbit residuals are estimated empirically. The first cheap experiment should use a small tanh RNN on a synthetic sinusoidal sequence and sweep input amplitude. Compare zero-state fixed-point solves against continuation warm starts, measuring iteration count, residual, and prediction error. The quantitative prediction is that cold starts lose convergence near a fold while continuation follows the orbit farther, and sigma_min(I-M) decreases toward zero as the leading multiplier approaches +1.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a readable scalar tanh-RNN periodic-orbit continuation MVP with fixed-point solving, pseudo-arclength correction, Floquet multiplier and singular-gap diagnostics, and a cold-start comparison. The checks confirmed the analytic fold at p=-0.207546 with multiplier +1, Jacobian error 2.2e-11, square-root gap scaling with observed exponent 0.456 versus predicted 0.5, and nearly constant iteration-times-gap products. Continuation tracked the connected orbit to within 3.8e-5 of the fold, turned around, and returned along the branch with residual below 7.7e-11; this demonstrates the mechanism, although cold starts did not diverge and instead selected a different stable branch.

Agent confidence: 8/10

Baseline: Zero-state fixed-point iteration converged at the final tested parameter with residual 1.95e-11 in 25 iterations, but selected a branch separated from continuation by 1.0484 state units.
Idea: Pseudo-arclength continuation tracked 66 orbit points, reached minimum p-fold distance 3.83e-5, turned and returned past the fold, with maximum residual 7.68e-11. Observed sigma_min(I-M) versus distance exponent was 0.456 (predicted 0.5); iteration-times-gap product ranged from 10.90 to 13.02.

Limitations: The experiment uses a scalar T=1 tanh recurrence rather than a multidimensional RNN with genuinely time-periodic forcing. It does not benchmark wall-clock inference speed, GPU execution, prediction error, randomized SVD, or a case where cold fixed-point iteration truly fails; the continuation trace approaches the fold but does not numerically cross to p below it because the chosen step size and tolerance stop 3.8e-5 short.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.21256) Sep 2, 2026

Ahead of the baseline (−35.0%) but not statistically significant (p=0.055, wins 6 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0014
Idea mean: 0.0009
Effect (Δ): -0.0005 (−35.0%; negative = idea better)
Wins: 6 / 8 paired seeds
p-value: 0.055 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±32.8%
Mechanism: Not confirmed ✗
Practical effect: Inconclusive
Baseline tuning: swept over 3 configs

Limitations:

The benchmark used the built-in 400-train/200-test reduced dynamics setting and 12 epochs rather than the larger default dataset. The correction was a practical fixed-point warm-start approximation, not full pseudo-arclength continuation across an externally swept forcing amplitude; no wall-clock inference-speed comparison or true near-fold failure case was tested.

How to run: python3 bench_experiment.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Failed ✗

Agent confidence: 9/10

Implemented periodic-orbit correction for the matched dynamics/rnn_small benchmark. Baseline and idea used the same GRU(3,64)+linear-head system and differed only in recurrent-state initialization/correction. The idea reduced mean test MSE from 0.00141983 to 0.00092240 with 6/8 paired wins, but permutation p=0.055, so it did not meet the required significant-win criterion; the trained Floquet signature was also not confirmed.

Baseline: Best lr=0.01; 8-seed mean test MSE 0.0014198302378645167, std 0.00039839114901047844.
Delta mean: -0.0004974290859536268
Idea: Best shared lr=0.01; 8-seed mean test MSE 0.0009224011519108899, std 0.00037536582303428516.
Idea wins: 6/8
Mechanism confirmed: false
P value: 0.055

Benchmark result