✗ Failed on benchmark 2026

Averaged Contractive State-Space Network

Implementation & benchmark of arXiv:2608.09319 — Averaging Principle and Pullback Attractor Convergence for McKean--Vlasov Stochastic Reaction--Diffusion Equations

Usefulness8/10

Difficulty6/10

Novelty7/10

Dynamics Stability Accuracy Memory Ssm Rnn Training dynamics Inference speedup Dynamical systems Control theory Stochastic processes Pde

Source paper: Averaging Principle and Pullback Attractor Convergence for McKean--Vlasov Stochastic Reaction--Diffusion Equations arXiv:2608.09319 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Construct a continuous-time SSM or neural ODE whose hidden-state dynamics use rapidly varying periodic parameters while enforcing contraction of the instantaneous Jacobian. In the high-frequency regime, replace the expensive oscillatory dynamics with an averaged SSM during long-horizon rollout; the averaging principle predicts finite-horizon trajectory convergence, while contraction predicts stable long-time behavior.

Formulas

$$\dot h_{\varepsilon}(t)=F\left(\frac{t}{\varepsilon},h_{\varepsilon}(t),x(t)\right),\qquad \dot{\bar h}(t)=\bar F\left(\bar h(t),x(t)\right),\qquad \bar F(h,x)=\frac{1}{T}\int_0^T F(s,h,x)\,ds.$$

$$\lim_{\varepsilon\to0}\sup_{0\leq t\leq L}\mathbb{E}\left\|h_{\varepsilon}(t)-\bar h(t)\right\|^2=0.$$

$$\mu_2\left(\partial_hF(s,h,x)\right)=\lambda_{\max}\left(\frac{\partial_hF(s,h,x)+\partial_hF(s,h,x)^{\mathsf T}}{2}\right)\leq-\alpha<0.$$

$$\|h(t)-\tilde h(t)\|\leq e^{-\alpha(t-t_0)}\|h(t_0)-\tilde h(t_0)\|,\qquad \operatorname{dist}(\mathcal{A}_{\varepsilon},\mathcal{A}_0)\to0.$$

Mathematical statement

Let $h(t)\in\mathbb{R}^{n}$ be the hidden state, $x(t)$ the input, $\varepsilon>0$ the fast time scale, and $F(s,h,x)$ a vector field periodic in phase $s$ with period $T$. The oscillatory system is $\dot h_{\varepsilon}=F(t/\varepsilon,h_{\varepsilon},x(t))$. Its averaged system is $\dot{\bar h}=\bar F(\bar h,x)$ with $\bar F(h,x)=T^{-1}\int_0^T F(s,h,x)\,ds$. The averaging result predicts $\sup_{0\leq t\leq L}\mathbb{E}\|h_{\varepsilon}(t)-\bar h(t)\|^2\to0$ for every fixed horizon $L$. A contraction certificate is $\mu_2(\partial_hF)\leq-\alpha<0$, where $\mu_2(J)=\lambda_{\max}((J+J^{\mathsf T})/2)$; this gives exponential forgetting of initial conditions at rate at least $\alpha$ and supports convergence of pullback attractors.

Implementation notes

Integrate this mechanism at the hidden-state transition of a continuous-time SSM, neural ODE, or liquid RNN. Use a phase-dependent parameterization such as $F(s,h,x)=A(s)h+\phi(B(s)h+Cx+b(s))$, with phase $s=t/\varepsilon$ and period $T=1$. During training, approximate the averaged vector field with quadrature: sample phases $s_j=j/M$ and compute $\bar F(h,x)\approx M^{-1}\sum_{j=0}^{M-1}F(s_j,h,x)$. Enforce contraction with spectral normalization or the penalty $[\mu_2(\partial_hF)+\alpha]_+^2$; estimate $\mu_2$ using automatic differentiation and a few power iterations. Run the fast model with phase updates at every solver step and the averaged model with one fixed transition. The first cheap experiment should use a width-64 neural ODE or linear-plus-tanh SSM on sequential MNIST, the copy-memory task, and a forced-sine long-horizon prediction task. Sweep $\varepsilon\in\{1,1/2,1/4,1/8,1/16\}$ and compare synchronized trajectories. The prediction is that fixed-horizon mean-square trajectory error decreases as $\varepsilon$ decreases, while perturbation errors decay with fitted log-slope no greater than $-\alpha$. After long warmup, the diameter of states from multiple initial conditions should decay approximately as $e^{-\alpha t}$; when the empirical matrix measure becomes positive, long-horizon equivalence should fail.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built and numerically verified a periodically modulated contractive state-space ODE with its exact averaged counterpart. Averaging error decreased nearly linearly with epsilon (log-log slope 0.928), the measured contraction boundary matched the predicted q=alpha=1.2 exactly, and perturbation decay matched the predicted -alpha slope to within 2e-7. The averaged rollout was 14.4x faster at epsilon=0.0625, confirming the mechanism in the toy setting, though not yet in a trained neural SSM.

Agent confidence: 9/10

Baseline: Phase-resolved oscillatory rollout: max trajectory errors for epsilon=[0.5,0.25,0.125,0.0625,0.03125] were [0.09089,0.05055,0.02678,0.01377,0.00699]; median runtime 0.7832 s.
Idea: Exact averaged rollout: median runtime 0.05431 s, 14.42x speedup; observed averaging-error log-log slope 0.9276; contraction boundary observed exactly at q=1.2; late perturbation slopes ranged from -1.2000000 to -1.2000002.

Limitations: This is a 2D linear toy ODE rather than a trained width-64 neural SSM or neural ODE. It uses an analytically known average, constant input, deterministic trajectories, and does not test sequential MNIST, copy-memory, learned parameters, stochastic inputs, non-symmetric Jacobians, or batched GPU inference.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.09319) Sep 2, 2026

Significantly WORSE than the tuned baseline (+3050.1%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0095
Idea mean: 0.2998
Effect (Δ): 0.2903 (+3050.1%; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±683.7%
Mechanism: Not confirmed ✗
Practical effect: Hurts
Baseline tuning: swept over 3 configs

Limitations:

Only the matched built-in dynamics track was tested; sequential and vision/tabular tracks were not run. The benchmark was reduced to n_train=400, n_test=160, and 5 epochs to fit the time budget after the initial larger run timed out. The implementation uses a discrete Euler recurrence and a simplified phase-modulated diagonal drift rather than a full continuous-time neural ODE solver.

How to run: python3 bench_stage2.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Failed ✗

Agent confidence: 9/10

Implemented an averaged contractive state-space recurrent model and compared it end-to-end against the matched bench rnn_small GRU on the structurally appropriate dynamics track. Across 8 paired seeds, the idea had MSE 0.29984 versus baseline 0.00952, with paired delta +0.29033 and permutation p=0.0081, so it was significantly worse. The trained-model mechanism signature observed negative matrix measure but only a noisy slope of 0.258 rather than the predicted approximately 1.0, and was therefore not confirmed.

Baseline: dynamics/rnn_small; best lr=0.01 from sweep [0.001, 0.003, 0.01]; 8-seed test MSE mean=0.0095184, std=0.0041543
Idea: averaged contractive Euler recurrence; best tested lr=0.01; 8-seed test MSE mean=0.2998441, std=0.0752087; paired delta=+0.2903256; permutation p=0.0081; observed trained-model averaging slope=0.2579; observed max mu2=-1.5681; mechanism confirmed=false

Benchmark result