# Averaged Contractive State-Space Network

- ID: 2040
- Canonical URL: https://synthcore.org/idea/2040/averaged-contractive-state-space-network
- API JSON: https://synthcore.org/api/idea/2040.json
- API Markdown: https://synthcore.org/api/idea/2040.md
- Verification status: failed_benchmark
- Source: [arXiv:2608.09319](https://arxiv.org/abs/2608.09319)
- Category: dynamics
- Solves: stability, accuracy, memory
- ML areas: ssm, rnn, training-dynamics, inference-speedup
- Math tags: dynamical-systems, control-theory, stochastic-processes, pde
- Ratings: usefulness 8/10; difficulty 6/10; novelty 7/10

## Idea description

Construct a continuous-time SSM or neural ODE whose hidden-state dynamics use rapidly varying periodic parameters while enforcing contraction of the instantaneous Jacobian. In the high-frequency regime, replace the expensive oscillatory dynamics with an averaged SSM during long-horizon rollout; the averaging principle predicts finite-horizon trajectory convergence, while contraction predicts stable long-time behavior.

## Mathematical statement

Let $h(t)\in\mathbb{R}^{n}$ be the hidden state, $x(t)$ the input, $\varepsilon>0$ the fast time scale, and $F(s,h,x)$ a vector field periodic in phase $s$ with period $T$. The oscillatory system is $\dot h_{\varepsilon}=F(t/\varepsilon,h_{\varepsilon},x(t))$. Its averaged system is $\dot{\bar h}=\bar F(\bar h,x)$ with $\bar F(h,x)=T^{-1}\int_0^T F(s,h,x)\,ds$. The averaging result predicts $\sup_{0\leq t\leq L}\mathbb{E}\|h_{\varepsilon}(t)-\bar h(t)\|^2\to0$ for every fixed horizon $L$. A contraction certificate is $\mu_2(\partial_hF)\leq-\alpha<0$, where $\mu_2(J)=\lambda_{\max}((J+J^{\mathsf T})/2)$; this gives exponential forgetting of initial conditions at rate at least $\alpha$ and supports convergence of pullback attractors.

## Key formulas

- $$\dot h_{\varepsilon}(t)=F\left(\frac{t}{\varepsilon},h_{\varepsilon}(t),x(t)\right),\qquad \dot{\bar h}(t)=\bar F\left(\bar h(t),x(t)\right),\qquad \bar F(h,x)=\frac{1}{T}\int_0^T F(s,h,x)\,ds.$$
- $$\lim_{\varepsilon\to0}\sup_{0\leq t\leq L}\mathbb{E}\left\|h_{\varepsilon}(t)-\bar h(t)\right\|^2=0.$$
- $$\mu_2\left(\partial_hF(s,h,x)\right)=\lambda_{\max}\left(\frac{\partial_hF(s,h,x)+\partial_hF(s,h,x)^{\mathsf T}}{2}\right)\leq-\alpha<0.$$
- $$\|h(t)-\tilde h(t)\|\leq e^{-\alpha(t-t_0)}\|h(t_0)-\tilde h(t_0)\|,\qquad \operatorname{dist}(\mathcal{A}_{\varepsilon},\mathcal{A}_0)\to0.$$

## Implementation notes

Integrate this mechanism at the hidden-state transition of a continuous-time SSM, neural ODE, or liquid RNN. Use a phase-dependent parameterization such as $F(s,h,x)=A(s)h+\phi(B(s)h+Cx+b(s))$, with phase $s=t/\varepsilon$ and period $T=1$. During training, approximate the averaged vector field with quadrature: sample phases $s_j=j/M$ and compute $\bar F(h,x)\approx M^{-1}\sum_{j=0}^{M-1}F(s_j,h,x)$. Enforce contraction with spectral normalization or the penalty $[\mu_2(\partial_hF)+\alpha]_+^2$; estimate $\mu_2$ using automatic differentiation and a few power iterations. Run the fast model with phase updates at every solver step and the averaged model with one fixed transition. The first cheap experiment should use a width-64 neural ODE or linear-plus-tanh SSM on sequential MNIST, the copy-memory task, and a forced-sine long-horizon prediction task. Sweep $\varepsilon\in\{1,1/2,1/4,1/8,1/16\}$ and compare synchronized trajectories. The prediction is that fixed-horizon mean-square trajectory error decreases as $\varepsilon$ decreases, while perturbation errors decay with fitted log-slope no greater than $-\alpha$. After long warmup, the diameter of states from multiple initial conditions should decay approximately as $e^{-\alpha t}$; when the empirical matrix measure becomes positive, long-horizon equivalence should fail.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: harms
- Verdict: Built and numerically verified a periodically modulated contractive state-space ODE with its exact averaged counterpart. Averaging error decreased nearly linearly with epsilon (log-log slope 0.928), the measured contraction boundary matched the predicted q=alpha=1.2 exactly, and perturbation decay matched the predicted -alpha slope to within 2e-7. The averaged rollout was 14.4x faster at epsilon=0.0625, confirming the mechanism in the toy setting, though not yet in a trained neural SSM.

### Mechanism check

- Verdict: Built and numerically verified a periodically modulated contractive state-space ODE with its exact averaged counterpart. Averaging error decreased nearly linearly with epsilon (log-log slope 0.928), the measured contraction boundary matched the predicted q=alpha=1.2 exactly, and perturbation decay matched the predicted -alpha slope to within 2e-7. The averaged rollout was 14.4x faster at epsilon=0.0625, confirming the mechanism in the toy setting, though not yet in a trained neural SSM.
- Confidence: 9/10
- Limitations: This is a 2D linear toy ODE rather than a trained width-64 neural SSM or neural ODE. It uses an analytically known average, constant input, deterministic trajectories, and does not test sequential MNIST, copy-memory, learned parameters, stochastic inputs, non-symmetric Jacobians, or batched GPU inference.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/769/bench_report.json)
- [bench_stage2.py](https://synthcore.org/code/769/bench_stage2.py)
- [experiment.py](https://synthcore.org/code/769/experiment.py)
- [report.md](https://synthcore.org/code/769/report.md)
- [report_bench_2026-09-01T101854.md](https://synthcore.org/code/769/report_bench_2026-09-01T101854.md)
- [results.json](https://synthcore.org/code/769/results.json)
- [Download all files as ZIP](https://synthcore.org/download/769)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.