# Characteristic-Root-Stable Delayed Recurrent Layer

- ID: 1081
- Canonical URL: https://synthcore.org/idea/1081/characteristic-root-stable-delayed-recurrent-layer
- API JSON: https://synthcore.org/api/idea/1081.json
- API Markdown: https://synthcore.org/api/idea/1081.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.17645](https://arxiv.org/abs/2607.17645)
- Category: dynamics
- Solves: stability, accuracy
- ML areas: rnn, ssm, training-dynamics, regularization
- Math tags: dynamical-systems, control-theory, spectral-theory, stochastic-processes
- Ratings: usefulness 8/10; difficulty 6/10; novelty 6/10

## Idea description

Build a recurrent layer whose feedback is explicitly filtered through a trainable distributed-delay kernel rather than an unconstrained one-step recurrence. At each update, use the local characteristic equation induced by the feedback gain and kernel Laplace transform to reject parameter settings with right-half-plane roots or to maintain a prescribed stability margin.

## Mathematical statement

For a linearization around an equilibrium, let \(z(t)\) be a perturbation satisfying \(\dot z(t)=a z(t)+b\int_0^\infty l(s)z(t-s)\,ds\), where \(a\) is the instantaneous Jacobian gain, \(b\) is delayed feedback gain, and \(l(s)\geq 0\) is normalized by \(\int_0^\infty l(s)ds=1\). The exponential ansatz \(z(t)=e^{\lambda t}\) produces \(\chi(\lambda)=\lambda-a-b\widehat l(\lambda)=0\), with \(\widehat l(\lambda)=\int_0^\infty e^{-\lambda s}l(s)ds\). Local stability requires every characteristic root to have negative real part. For a discrete delay, \(l(s)=\delta(s-\tau)\), and a Hopf boundary satisfies \(a+b\cos(\omega\tau)=0\) and \(\omega+b\sin(\omega\tau)=0\).

## Key formulas

- $$\dot z(t)=a z(t)+b\int_0^\infty l(s)z(t-s)\,ds,\qquad \widehat l(\lambda)=\int_0^\infty e^{-\lambda s}l(s)\,ds,$$
- $$\chi(\lambda)=\lambda-a-b\widehat l(\lambda)=0,\qquad \text{stable if }\max_{\chi(\lambda)=0}\operatorname{Re}(\lambda)<0,$$
- $$l(s)=\delta(s-\tau):\qquad i\omega-a-b e^{-i\omega\tau}=0,$$
- $$a+b\cos(\omega\tau)=0,\qquad \omega+b\sin(\omega\tau)=0,\qquad \omega^2=b^2-a^2.$$

## Implementation notes

Use this at the recurrent-state update of a continuous-time RNN or state-space model: \(\dot h=F_\theta(h,x)+Bq\), where \(q(t)=\int l_\phi(s)h(t-s)ds\). Begin with a discrete delay buffer or a two- to four-stage Erlang kernel, which can be implemented as cascaded first-order states. At each minibatch, estimate the instantaneous scalar gain \(a\) and delayed gain \(b\) from Jacobian-vector products of the recurrent map. Numerically search for the rightmost roots of \(\chi(\lambda)=\lambda-a-b\widehat l_\phi(\lambda)\). Add \(\rho\max(0,\operatorname{Re}\lambda_*+m)^2\) to the loss, where \(m>0\) is a desired margin, or reject an optimizer step when \(\operatorname{Re}\lambda_*>-m\). The characteristic equation and Laplace transform are analytical inputs from the paper; gains and dominant roots are estimated locally. First experiment: compare an unconstrained delayed RNN with the penalty on copy-memory and sequential-MNIST. Sweep feedback gain and delay. The predicted signature is a sharp trajectory-divergence boundary when the measured rightmost root crosses zero, with the oscillation onset frequency within 20% of \(\omega=\sqrt{b^2-a^2}\). In the stable regime, perturbations should decay approximately as \(e^{\operatorname{Re}(\lambda_*)t}\).

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: harms
- Verdict: Built a Lambert-W characteristic-root verifier for the discrete-delay equation plus Euler trajectory and toy training comparisons. Across four delays, observed stability boundaries exactly matched the analytical Hopf boundaries; predicted frequencies matched within 1.9%, and measured decay matched the dominant-root real part within 15%. The positive-feedback sweep crossed from decay to growth at b=-a=0.5 with measured slopes matching root predictions to below 0.001%; the constrained toy ended stable (root real part -0.0559) while the unconstrained baseline remained unstable (+0.6848).

### Mechanism check

- Verdict: Built a Lambert-W characteristic-root verifier for the discrete-delay equation plus Euler trajectory and toy training comparisons. Across four delays, observed stability boundaries exactly matched the analytical Hopf boundaries; predicted frequencies matched within 1.9%, and measured decay matched the dominant-root real part within 15%. The positive-feedback sweep crossed from decay to growth at b=-a=0.5 with measured slopes matching root predictions to below 0.001%; the constrained toy ended stable (root real part -0.0559) while the unconstrained baseline remained unstable (+0.6848).
- Confidence: 9/10
- Limitations: Only a scalar discrete-delay linearized system and a tiny synthetic regression were tested; no vector-valued delayed RNN, Erlang kernels, sequential-MNIST, CUDA training, learned kernel parameters, Jacobian-vector-product estimation, or wall-clock/FLOP comparison was evaluated. The stability projection uses an analytically sufficient scalar constraint rather than a general numerical root-penalty optimizer.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_delayed_rnn.py](https://synthcore.org/code/194/bench_delayed_rnn.py)
- [bench_report.json](https://synthcore.org/code/194/bench_report.json)
- [report.md](https://synthcore.org/code/194/report.md)
- [report_bench_2026-08-30T201631.md](https://synthcore.org/code/194/report_bench_2026-08-30T201631.md)
- [results.json](https://synthcore.org/code/194/results.json)
- [verify_delayed_rnn.py](https://synthcore.org/code/194/verify_delayed_rnn.py)
- [Download all files as ZIP](https://synthcore.org/download/194)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.