Block-Hurwitz barrier for polynomial state-space layers
Source paper: On the Coefficients of Hurwitz-Type Matrix Polynomials arXiv:2608.30089 ⓘ · analyzed Sep 1, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Constrain the learned coefficients of a high-order linear recurrent or state-space layer using the block Hurwitz matrix associated with its matrix characteristic polynomial. Penalize near-singular Hurwitz blocks and, for degrees two and three, optionally enforce positive leading Hurwitz determinants; use companion-matrix eigenvalues as the definitive stability check rather than trusting determinant positivity at degree four or above.
Formulas
Mathematical statement
The paper writes a monic matrix polynomial as \(\mathbf f_n(z)=I_qz^n+A_1z^{n-1}+\cdots+A_n\), with \(A_0:=I_q\), and decomposes it as \(\mathbf f_n(z)=\mathbf h_n(z^2)+z\mathbf g_n(z^2)\). For zero-based block indices \(i,j\in\{0,\ldots,n-1\}\), the displayed block Hurwitz matrix can be represented as \(H_f^{(n)}=[A_{2j-i}]_{i,j=0}^{n-1}\), where \(A_r=0\) for \(r<0\) or \(r>n\); its first rows are \([A_0,A_2,A_4,\ldots]\), \([0,A_1,A_3,\ldots]\), and \([0,A_0,A_2,\ldots]\). The paper relates this matrix to block Hankel and Schur-complement constructions and studies positivity of its coefficient determinants, proving the claimed positivity only through degree three and giving a degree-four counterexample. For a neural layer, define the companion matrix \(C_f\) of \(\mathbf f_n\) by \(\dot x=C_fx\), with block rows \([0,I_q,0,\ldots]\) and final row \([-A_n,-A_{n-1},\ldots,-A_1]\); Hurwitz stability means \(\max\operatorname{Re}\lambda(C_f)<0\). Use \(H_f^{(n)}\) as a conditioning barrier and the companion eigenvalues as the ground-truth stability metric.
Implementation notes
Integrate this into a continuous-time high-order RNN or SSM whose trainable dynamics are represented by coefficient blocks \(A_1,\ldots,A_n\), rather than directly by an unconstrained dense companion matrix. First use \(n=2\) or \(n=3\), with state dimension \(nq\). At every training step, construct \(H\) from \(H_{ij}=A_{2j-i}\) using zero-based block indices and zero padding. Form \(G=H^*H+\varepsilon I\), with \(\varepsilon=10^{-5}\), and add \(\lambda_H[-\log\det G+\log\det G_0]\), where \(G_0\) is a detached reference scale; alternatively use \(\lambda_H\sum_k\operatorname{softplus}(\tau-\sigma_k(H))\). For degrees two and three, also compute leading principal block determinants \(\Delta_k=\det H_{0:k,0:k}\) and add \(\lambda_\Delta\operatorname{softplus}(\tau-\operatorname{Re}\Delta_k)\) only when coefficients are real or Hermitian-compatible. Do not impose this determinant rule for degree four or higher because the paper explicitly supplies a counterexample. Independently build \(C_f\), estimate its rightmost eigenvalue every 50 steps, and reject or clip updates producing \(\alpha(C_f)>-\rho\). The paper supplies the block construction; singular values, determinants, eigenvalues, and barrier gradients are computed numerically. The first experiment should compare a width-matched vanilla linear SSM and this regularized version on sequential MNIST or synthetic long-range copying, using \(n=3,q=16\). Measure training loss at equal FLOPs, the fraction of runs with exploding hidden states, gradient-norm variance, and final companion spectral abscissa. Success is lower loss and fewer unstable runs without materially increasing per-step cost.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.