✗ Failed on benchmark 2026

Characteristic-Root-Stable Delayed Recurrent Layer

Implementation & benchmark of arXiv:2607.17645 — Macroscopic Multistability and Bifurcations in Theta-Neuron Networks with Distributed Delays

Usefulness8/10

Difficulty6/10

Novelty6/10

Dynamics Stability Accuracy Rnn Ssm Training dynamics Regularization Dynamical systems Control theory Spectral theory Stochastic processes

Source paper: Macroscopic Multistability and Bifurcations in Theta-Neuron Networks with Distributed Delays arXiv:2607.17645 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Build a recurrent layer whose feedback is explicitly filtered through a trainable distributed-delay kernel rather than an unconstrained one-step recurrence. At each update, use the local characteristic equation induced by the feedback gain and kernel Laplace transform to reject parameter settings with right-half-plane roots or to maintain a prescribed stability margin.

Formulas

$$\dot z(t)=a z(t)+b\int_0^\infty l(s)z(t-s)\,ds,\qquad \widehat l(\lambda)=\int_0^\infty e^{-\lambda s}l(s)\,ds,$$

$$\chi(\lambda)=\lambda-a-b\widehat l(\lambda)=0,\qquad \text{stable if }\max_{\chi(\lambda)=0}\operatorname{Re}(\lambda)<0,$$

$$l(s)=\delta(s-\tau):\qquad i\omega-a-b e^{-i\omega\tau}=0,$$

$$a+b\cos(\omega\tau)=0,\qquad \omega+b\sin(\omega\tau)=0,\qquad \omega^2=b^2-a^2.$$

Mathematical statement

For a linearization around an equilibrium, let $z(t)$ be a perturbation satisfying $\dot z(t)=a z(t)+b\int_0^\infty l(s)z(t-s)\,ds$, where $a$ is the instantaneous Jacobian gain, $b$ is delayed feedback gain, and $l(s)\geq 0$ is normalized by $\int_0^\infty l(s)ds=1$. The exponential ansatz $z(t)=e^{\lambda t}$ produces $\chi(\lambda)=\lambda-a-b\widehat l(\lambda)=0$, with $\widehat l(\lambda)=\int_0^\infty e^{-\lambda s}l(s)ds$. Local stability requires every characteristic root to have negative real part. For a discrete delay, $l(s)=\delta(s-\tau)$, and a Hopf boundary satisfies $a+b\cos(\omega\tau)=0$ and $\omega+b\sin(\omega\tau)=0$.

Implementation notes

Use this at the recurrent-state update of a continuous-time RNN or state-space model: $\dot h=F_\theta(h,x)+Bq$, where $q(t)=\int l_\phi(s)h(t-s)ds$. Begin with a discrete delay buffer or a two- to four-stage Erlang kernel, which can be implemented as cascaded first-order states. At each minibatch, estimate the instantaneous scalar gain $a$ and delayed gain $b$ from Jacobian-vector products of the recurrent map. Numerically search for the rightmost roots of $\chi(\lambda)=\lambda-a-b\widehat l_\phi(\lambda)$. Add $\rho\max(0,\operatorname{Re}\lambda_*+m)^2$ to the loss, where $m>0$ is a desired margin, or reject an optimizer step when $\operatorname{Re}\lambda_*>-m$. The characteristic equation and Laplace transform are analytical inputs from the paper; gains and dominant roots are estimated locally. First experiment: compare an unconstrained delayed RNN with the penalty on copy-memory and sequential-MNIST. Sweep feedback gain and delay. The predicted signature is a sharp trajectory-divergence boundary when the measured rightmost root crosses zero, with the oscillation onset frequency within 20% of $\omega=\sqrt{b^2-a^2}$. In the stable regime, perturbations should decay approximately as $e^{\operatorname{Re}(\lambda_*)t}$.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built a Lambert-W characteristic-root verifier for the discrete-delay equation plus Euler trajectory and toy training comparisons. Across four delays, observed stability boundaries exactly matched the analytical Hopf boundaries; predicted frequencies matched within 1.9%, and measured decay matched the dominant-root real part within 15%. The positive-feedback sweep crossed from decay to growth at b=-a=0.5 with measured slopes matching root predictions to below 0.001%; the constrained toy ended stable (root real part -0.0559) while the unconstrained baseline remained unstable (+0.6848).

Agent confidence: 9/10

Baseline: Toy final MSE 0.01753; feedback b=2.34995; rightmost root real part +0.68481 (unstable).
Idea: Toy final MSE 0.01711; projected feedback b=0.419997; rightmost root real part -0.05587 (stable). Hopf boundary relative error 0 across tau=0.5,1,2,3; frequency error 0.8%-1.9%; decay-slope relative error 14.9%.

Limitations: Only a scalar discrete-delay linearized system and a tiny synthetic regression were tested; no vector-valued delayed RNN, Erlang kernels, sequential-MNIST, CUDA training, learned kernel parameters, Jacobian-vector-product estimation, or wall-clock/FLOP comparison was evaluated. The stability projection uses an analytically sufficient scalar constraint rather than a general numerical root-penalty optimizer.

How to run: python3 verify_delayed_rnn.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.17645) Sep 2, 2026

Significantly WORSE than the tuned baseline (+502.2%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0
Idea mean: 0
Effect (Δ): 0 (+502.2%; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±164.4%
Mechanism: Not confirmed ✗
Practical effect: Hurts
Baseline tuning: swept over 3 configs

Limitations:

Only the built-in controlled-pendulum dynamics track was tested; no sequential-MNIST, learned kernel-delay sweep, JVP penalty, wall-clock comparison, or longer-horizon evaluation was performed.

How to run: python3 bench_delayed_rnn.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 30, 2026 latest Failed ✗

Agent confidence: 10/10

The characteristic-root-inspired delayed recurrent layer was implemented and evaluated against the matched rnn_small GRU baseline on the controlled-pendulum dynamics track. The idea was significantly worse: mean test MSE 4.17733e-05 versus 6.93655e-06 for baseline, paired delta +3.48367e-05, permutation p=0.0081. The trained-model mechanism signature was also not confirmed because the measured spectral radius was 1.0155 and the decay prediction relative error was 170%.

Baseline: mean test MSE 6.936550448699563e-06; best lr 0.01
Idea: mean test MSE 4.17732610458188e-05; best lr 0.01
Paired delta: 3.4836710597119236e-05; p=0.0081; 0/8 idea wins

Benchmark result