Unverified 2026

Full-block IQC certificates for stable RNNs

Implementation & benchmark of arXiv:2607.23599 — Existence of stable Lur'e systems for which the O'Shea-Zames-Falb stability test fails

Usefulness6/10

Difficulty7/10

Novelty6/10

Dynamics Stability Generalization Rnn Training dynamics Control theory Dynamical systems Convex analysis Linear algebra

Source paper: Existence of stable Lur'e systems for which the O'Shea-Zames-Falb stability test fails arXiv:2607.23599 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Model an RNN as a linear state-space system in feedback with its slope-restricted activation, then search for a finite-horizon IQC multiplier instead of relying only on a spectral-radius or OZF-style condition. Penalize or reject parameter settings for which the strict IQC/LMI certificate has insufficient margin, yielding a directly testable stability criterion for long unrolled sequences.

Formulas

$$0\leq\frac{\phi(x_{1})-\phi(x_{2})}{x_{1}-x_{2}}\leq\kappa\qquad\text{for all }x_1,x_2\in\mathbb{R}.$$

$$\sum_{t=0}^{T} z_t^{\top}\Pi z_t\geq 0,\qquad z=\Psi\begin{bmatrix}u\\y\end{bmatrix}.$$

$$\begin{pmatrix}I&0\\\mathcal{A}&\mathcal{B}\\\mathcal{C}&\mathcal{D}\end{pmatrix}^{\!\top}\begin{pmatrix}-\mathcal{X}&0&0\\0&\mathcal{X}&0\\0&0&\Pi\end{pmatrix}\begin{pmatrix}I&0\\\mathcal{A}&\mathcal{B}\\\mathcal{C}&\mathcal{D}\end{pmatrix}\prec0,\qquad \mathcal{X}\succeq0,\quad \operatorname{tr}(\mathcal{X})=1.$$

$$h_{t+1}=Ah_t+Bu_t+B_\phi\phi(C_\phi h_t+D_\phi u_t),\qquad y_t=C_yh_t+D_yu_t.$$

$$M(\mathcal X,\Pi)+\rho I\preceq0,\qquad \mathcal L_{\mathrm{IQC}}=\lambda\max(0,\rho_0-\rho)^2.$$

Mathematical statement

The paper assumes a memoryless nonlinearity $\phi$ satisfying the slope restriction $0\leq(\phi(x_1)-\phi(x_2))/(x_1-x_2)\leq\kappa$, where $\kappa$ is the activation slope bound. For an RNN, let $G$ be the stable linear state-space system with state $h_t$, input $u_t$, output $y_t$, and let $\Delta$ represent the activation feedback. A lifting filter $\Psi$ maps the input-output sequence to $z_t$, and $\Pi\in\mathbb{S}^{2N\times 2N}$ is a multiplier such that every admissible $\Delta$ satisfies the hard IQC $\sum_{t=0}^{T}z_t^\top\Pi z_t\geq0$ for every finite horizon $T$. Theorem 3 states that stability follows if there is a symmetric matrix $\mathcal{X}\in\mathbb{S}^{(n_\Psi+n)\times(n_\Psi+n)}$ with $\mathcal{X}\succeq0$, $\operatorname{tr}(\mathcal{X})=1$, and $\begin{pmatrix}I&0\\\mathcal{A}&\mathcal{B}\\\mathcal{C}&\mathcal{D}\end{pmatrix}^{\!\top}\begin{pmatrix}-\mathcal{X}&0&0\\0&\mathcal{X}&0\\0&0&\Pi\end{pmatrix}\begin{pmatrix}I&0\\\mathcal{A}&\mathcal{B}\\\mathcal{C}&\mathcal{D}\end{pmatrix}\prec0$. Here $n$ is the RNN state dimension, $n_\Psi$ is the lifting-filter state dimension, and $(\mathcal{A},\mathcal{B},\mathcal{C},\mathcal{D})$ is the augmented realization of the RNN linear dynamics cascaded with $\Psi$. The strict negative-definiteness margin is the certificate quantity to maximize.

Implementation notes

1. Integration point: apply this to a tanh or ReLU RNN, or to a residual sequence block, during architecture search or periodically during training. Write the cell as $h_{t+1}=Ah_t+Bu_t+B_\phi v_t$, $v_t=\phi(C_\phi h_t+D_\phi u_t)$, with output $y_t=C_yh_t+D_yu_t$. For tanh use $\kappa=1$; for ReLU use the sector bound $0\leq\phi'\leq1$. 2. Pseudocode: construct the augmented state-space matrices $(\mathcal A,\mathcal B,\mathcal C,\mathcal D)$ from the RNN matrices and a short FIR lifting filter $\Psi$ with memory $N$. Choose a sector-valid symmetric multiplier $\Pi$, solve an SDP for $\mathcal X\succeq0$ and $\operatorname{tr}(\mathcal X)=1$, and maximize $\rho$ subject to $M(\mathcal X,\Pi)+\rho I\preceq0$, where $M$ is the left-hand side of the IQC inequality. Add $\lambda\max(0,\rho_0-\rho)^2$ to the network loss, or reject optimizer updates with $\rho<\rho_0$. 3. Compute the augmented matrices and LMI exactly; estimate only current Jacobian blocks if the architecture is nonlinear in its parameters. For an MVP, keep $\Pi$ fixed to a static sector multiplier, then compare it against an optimized full-block/FIR multiplier. 4. Test a two-layer tanh RNN on adding and permuted sequential MNIST, comparing spectral normalization and gradient clipping. Track $\rho$, exploding-gradient frequency, hidden-state norms over 1,000-step rollouts, validation accuracy, and SDP overhead. Success means fewer unstable long rollouts and improved long-horizon accuracy at matched parameter count and comparable training cost.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.