Path-complete stable routed SSM
Implementation & benchmark of arXiv:2607.00637 — Iterative graph lifting for automatic design of path-complete stability certificates
Source paper: Iterative graph lifting for automatic design of path-complete stability certificates arXiv:2607.00637 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace a single shared quadratic stability constraint in a routed state-space model with a path-complete family of quadratic certificates indexed by a small graph. During architecture search or training, identify bottleneck certificate nodes whose transition inequalities are nearly tight, split only those nodes, and re-solve the certificate problem. This should permit larger per-mode state transitions than a common Lyapunov matrix while retaining bounded hidden-state dynamics for arbitrary router sequences.
Formulas
Mathematical statement
For a finite set of mode matrices \(\mathcal A=\{A_1,\ldots,A_M\}\subset\mathbb R^{n\times n}\), the joint spectral radius is \(\rho(\mathcal A)=\lim_{k\to\infty}\max_{\sigma\in\langle M\rangle^k}\|A_{\sigma_k}\cdots A_{\sigma_1}\|^{1/k}\). A path-complete graph is \(G=(S,E)\), where each edge \((a,b,i)\in E\) means that mode \(A_i\) maps certificate node \(a\) to node \(b\). A path-complete Lyapunov family \(\{V_a\}_{a\in S}\) satisfies \(V_b(A_i x)\le V_a(x)\) for every edge and every \(x\). Use quadratic functions \(V_a(x)=x^\top P_a x\), with \(P_a\succ0\), giving the semidefinite inequalities \(\gamma^2P_a\succeq A_i^\top P_bA_i\). Feasibility implies \(\rho(\mathcal A)\le\gamma\); choosing \(\gamma<1\) certifies contraction. The paper's forward lift splits a node \(v\) into copies \(v^{(w)}\) indexed by \(w\in\mathrm{Post}(v)\), while copying the old matrix: \(\tilde P_a=P_a\) for \(a\ne v\) and \(\tilde P_{v^{(w)}}=P_v\). Hence every old feasible solution remains feasible at the same \(\gamma\), so refinement cannot worsen the certificate. In the neural adaptation, \(A_i\) is the trainable state-transition matrix used when router state or expert \(i\) is selected, and \(P_a\) is an auxiliary certificate matrix; the certificate is periodically recomputed rather than differentiated through at every minibatch.
Implementation notes
(1) Integration point: use a linear recurrent or state-space backbone inside a small language model or sequence model. Let the hidden state be \(h_t\in\mathbb R^n\), let a router choose mode \(i_t\in\{1,\ldots,M\}\), and update \(h_{t+1}=A_{i_t}h_t+B_{i_t}u_t\). Apply the certificate only to the homogeneous transition \(A_i h_t\); keep the input/output projections and nonlinear readout unconstrained. Initialize a graph with one node and edges labeled by every mode, or with a manually chosen small graph. (2) Pseudocode: repeat every K optimizer steps: collect current A[i]; solve min gamma subject to P[a] >= eps*I, trace(P[0])=n, gamma^2*P[a] - A[i].T@P[b]@A[i] >= 0 for every edge; compute slack[a,b,i] = lambda_min(gamma^2*P[a] - A[i].T@P[b]@A[i]); compute score[a] = sum_edges exp(-slack[a,b,i]/tau); if max(score) exceeds threshold, split the highest-score node v into clones indexed by its successor nodes, copy P[v] into every clone, redirect the local incoming and outgoing edges according to the lifted graph, and re-solve. During network training either project each \(A_i\) back into the feasible set after an optimizer step, or add a penalty \(\lambda\sum_e\|[A_i^\top P_bA_i-\gamma^2P_a]_+\|_F^2\) between exact SDP refreshes. (3) The SDP solver computes \(P_a\) and the smallest certified \(\gamma\); the eigenvalue slack and node scores are computed directly from the solved matrices. Estimate the empirical growth rate separately by running random and adversarial router sequences and measuring \(\|h_T\|/\|h_0\|\). If full-dimensional SDPs are too expensive, use state spaces of dimension 32 to 128, diagonal or low-rank-plus-diagonal \(P_a\), and refresh only every 500 to 2000 training steps. (4) First experiment: compare a one-layer routed SSM with shared quadratic \(P\), a fixed multi-node path-complete graph, and the adaptive lifting method on CIFAR-10 sequential pixels or the WikiText-2 language-modeling task. Use identical parameter counts and transition spectral-radius initialization. Measure maximum hidden-state norm over long random and adversarial routing sequences, certified \(\gamma\), validation loss, gradient-norm variance, and wall-clock overhead. A successful result is a substantially smaller certified \(\gamma\) or stable training at transition matrices whose common-\(P\) baseline is infeasible, with no validation-loss degradation and fewer graph nodes than a uniformly expanded history graph.
Verification
Stage 1 · Toy mechanism gate: Passed ✓
Stage 2 · Mechanism transferred to benchmark: Not tested
Stage 2 · Practical benchmark result: Not run
Methodology: Toy-system gate first; the benchmark stage runs only after a pass. How verification works
Stage 1 — Mechanism check agent confidence 7/10
Built a self-contained path-complete quadratic-certificate MVP with explicit positive-definiteness constraints, a mode-memory path-complete graph, and a forward-lift inheritance check. On the fixed two-mode non-normal system, the certified bound improved from gamma=1.9423 for a common quadratic certificate to gamma=1.4161 for the two-node path-complete certificate, while the lift retained positive slack of 0.6792. This is a promising stability-bound improvement, although both bounds exceed 1 and therefore do not certify contraction; adaptive node selection and neural training were not implemented.
- Agent confidence
- 7/10
- Baseline
- Common quadratic certificate: gamma=1.9422939746, feasible, minimum slack=-1.16e-11.
- Idea
- Two-node mode-memory path-complete certificate: gamma=1.4160808043, feasible, minimum slack=-3.43e-10; forward lift: 3 nodes, minimum slack=0.6791544249; empirical random-routing maximum asymptotic rate=1.2980124644 and maximum norm over 80 steps=1.154e9.
Limitations: The certificate optimization is an approximate scipy SLSQP fallback rather than a formal SDP solver. The experiment uses 2x2 synthetic transitions only; it does not test a neural sequence model, input-driven dynamics, adaptive bottleneck scoring during training, wall-clock training overhead, larger state dimensions, or adversarially optimized routing.
How to run: python3 experiment.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Artifacts
- experiment.py 3.9 KB View Raw
- report.md 1.6 KB View
- results.json 0.5 KB View Raw