Unverified 2026

Delay-Gain Certified Recurrent Block

Usefulness7/10

Difficulty7/10

Novelty7/10

Dynamics Stability Accuracy Rnn Ssm Optimizer Training dynamics Control theory Dynamical systems Optimization Linear algebra Numerical analysis

Source paper: Performance Analysis of Time-Delay Systems under External Perturbations Using Output-to-Output Gain arXiv:2608.28969 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace an unconstrained recurrent or state-space update with a delayed continuous-time hidden-state block and constrain its local closed-loop Jacobian using an output-to-output dissipativity LMI. The certificate bounds amplification from external perturbations, such as corrupted observations, injected hidden-state noise, or delayed-input errors, to the task output. Training rejects or penalizes parameter updates for which the certified gain becomes too large.

Formulas

$$\dot h(t)=A h(t)+A_1h(t-h_u)+A_2h(t-h_y)+B r(t),\qquad p(t)=C_p h(t)+D_p r(t),\quad q(t)=C_r h(t)+D_r r(t).$$

$$\begin{bmatrix}A^{\mathsf T}P+PA&PB\\B^{\mathsf T}P&0\end{bmatrix}-\begin{bmatrix}C_r^{\mathsf T}\\D_r^{\mathsf T}\end{bmatrix}\begin{bmatrix}C_r&D_r\end{bmatrix}+\gamma\begin{bmatrix}C_p^{\mathsf T}\\D_p^{\mathsf T}\end{bmatrix}\begin{bmatrix}C_p&D_p\end{bmatrix}\preceq0,\qquad P\succeq0,\ \gamma>0.$$$$

$$\dot V(h_t)+\|q(t)\|_2^2-\gamma\|p(t)\|_2^2\le0\ \Longrightarrow\ \int_0^T\|q(t)\|_2^2dt\le\gamma\int_0^T\|p(t)\|_2^2dt+V(h_0).$$

$$\min_{P\succeq0,\,\gamma}\ \gamma\quad\text{subject to}\quad \operatorname{LMI}_{\mathrm{LKF}}(A,A_1,A_2,B,C_p,D_p,C_r,D_r,h_u,h_y,P,\gamma)\preceq0.$$$$

Mathematical statement

Consider the linearized delayed hidden-state model $\dot h(t)=A h(t)+A_1h(t-h_u)+A_2h(t-h_y)+B r(t)$, with performance output $p(t)=C_p h(t)+D_p r(t)$ and perturbation-monitor output $q(t)=C_r h(t)+D_r r(t)$. Here $h$ is the hidden state, $r$ is an external perturbation, $h_u,h_y\ge0$ are independent delays, and the matrices are Jacobians or output maps. The paper searches for a positive semidefinite storage matrix $P$ and gain parameter $\gamma>0$ satisfying a delay-dependent Lyapunov–Krasovskii LMI. In the delay-free or Padé-augmented case, the core dissipativity condition is $\begin{bmatrix}A^{\mathsf T}P+PA&PB\\B^{\mathsf T}P&0\end{bmatrix}-\begin{bmatrix}C_r^{\mathsf T}\\D_r^{\mathsf T}\end{bmatrix}\begin{bmatrix}C_r&D_r\end{bmatrix}+\gamma\begin{bmatrix}C_p^{\mathsf T}\\D_p^{\mathsf T}\end{bmatrix}\begin{bmatrix}C_p&D_p\end{bmatrix}\preceq0$. This implies a dissipation inequality $\dot V+\|q\|^2-\gamma\|p\|^2\le0$, so $\gamma$ bounds the relevant output-to-output energy amplification under the paper's convention. For a nonlinear network this is a local or region-wise certificate unless Jacobian bounds are established over the full activation domain.

Implementation notes

1. Integration point: implement a continuous-time recurrent or state-space layer, $\dot h=f_\theta(h,h_{\mathrm{delayed}},x)$, with FIFO buffers for independently delayed input and observation channels. At training checkpoints, linearize the hidden dynamics around minibatch trajectories and obtain $A,A_1,A_2,B$ using automatic differentiation. Let $p$ be logits or task features and $q$ be the injected-noise output. Initially use a linear recurrent core with a nonlinear readout so the certificate is exact for the core. 2. Pseudocode: simulate the delayed buffers; compute the Jacobian matrices; assemble a first-order Padé augmented state for each delay pair, or call a delay-dependent LKF semidefinite program; solve for the smallest feasible $\gamma$; add $\lambda\max(0,\gamma-\gamma_{\max})^2$ to the loss, or reject an optimizer step when the SDP is infeasible. Use bisection over $\gamma$ and warm-start $P$ between checkpoints. 3. Computed from the paper: the storage-function dissipation inequality, LMI certificate, and certified gain bound. Estimated empirically: local Jacobians, effective delays, and whether the certificate holds across sampled states; global certification requires known Jacobian bounds over the activation region. 4. First cheap experiment: compare a vanilla delayed GRU or linear SSM, a spectral-normalized SSM, and this certified block on sequential MNIST or a delayed-copy task with additive hidden-state perturbations. Sweep the delay $h$ and perturbation amplitude. The quantitative prediction is a sharp feasibility boundary $h_c$: below $h_c$, the SDP returns finite $\gamma$ and measured amplification remains below $\sqrt{\gamma}$; above $h_c$, feasibility is lost or $\gamma$ rises rapidly. For the Padé approximation, the measured boundary should agree with direct delayed simulation within approximately 20%, and tightening the LMI objective should monotonically reduce measured perturbation amplification.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.