# Controllability-Regularized State-Space Layer

- ID: 720
- Canonical URL: https://synthcore.org/idea/720/controllability-regularized-state-space-layer
- API JSON: https://synthcore.org/api/idea/720.json
- API Markdown: https://synthcore.org/api/idea/720.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.10956](https://arxiv.org/abs/2607.10956)
- Category: dynamics
- Solves: stability, accuracy, generalization
- ML areas: ssm, rnn, training-dynamics, regularization
- Math tags: control-theory, linear-algebra, dynamical-systems, optimization
- Ratings: usefulness 7/10; difficulty 5/10; novelty 6/10

## Idea description

Replace an unconstrained latent transition in an SSM or recurrent block by quiver data (alpha,gamma), where alpha evolves the latent state and gamma injects token or feature inputs. Add a differentiable penalty that detects eigenmodes of alpha not reached from gamma, preventing dead latent directions and improving long-context signal propagation. The paper’s exact open condition becomes a practical regularizer rather than a hard architectural constraint.

## Mathematical statement

The paper uses vector spaces A and W, an endomorphism alpha in Hom(A,A), and an input map gamma:W to A. Its key condition is Im(alpha-lambda I_A)+Im(gamma)=A for every lambda in the algebraic closure of the base field. Here I_A is the identity on the latent state space, lambda ranges over eigenvalues of alpha, and Im denotes image. Equivalently, the concatenated matrix [alpha-lambda I_A, gamma] has full row rank dim(A); this is the PBH controllability criterion. The paper also identifies the failure locus Z={ (alpha,j): there exist lambda and nonzero v with j(v)=0 and alpha v=lambda v }, meaning that an eigenmode is invisible to the output map j:A to U. For a neural SSM, gamma is the input injection and j is the readout. Penalize small singular values of these PBH matrices at eigenvalues of alpha. An eigenvalue detached from gradients may be used when computing the penalty for numerical stability.

## Key formulas

- $$\operatorname{Im}(\alpha-\lambda\cdot\operatorname{Id}_{A})+\operatorname{Im}(\gamma)=A\quad\text{for all }\lambda\in\overline{{\mathbbm{k}}}.$$
- $$Z=\left\{(\alpha,j)\in V_{\mp,n,d}:\exists\lambda\in\overline{{\mathbbm{k}}},v\in A\text{ such that }j(v)=0\text{ and }\alpha v=\lambda v\right\}.$$
- $$\mathcal{C}(\alpha,\gamma)=\begin{bmatrix}\gamma&\alpha\gamma&\alpha^{2}\gamma&\cdots&\alpha^{d-1}\gamma\end{bmatrix},\qquad d=\dim A.$$
- $$\mathcal{L}_{\mathrm{ctrl}}=\sum_{\lambda\in\operatorname{spec}(\alpha)}\operatorname{softplus}\!\left(\tau-\sigma_{\min}\!\left([\alpha-\lambda I_A\ \ \gamma]\right)\right)+\eta\,\operatorname{softplus}\!\left(\tau-\sigma_{\min}\!\left(\begin{bmatrix}\alpha-\lambda I_A\\ j\end{bmatrix}\right)\right).$$

## Implementation notes

(1) Integration point: modify the transition and input-injection matrices of a discrete-time SSM or recurrent layer. For token features u_t in R^m and latent state h_t in R^d, use h_{t+1}=rho(alpha h_t+gamma u_t) and y_t=j h_t. Keep the rest of the neural block unchanged so the comparison isolates the regularizer. (2) Pseudocode: initialize alpha in R^{dxd}, gamma in R^{dxm}, and j in R^{rxd}; at each optimization step compute the task loss from the recurrence. Every K steps, calculate lam=eigvals(alpha.detach()). For each lam, form M=concat(alpha-lam*I,gamma) and compute its smallest singular value s. Add softplus(tau-s) to L_ctrl. Optionally form N=vertical_stack(alpha-lam*I,j) and add eta*softplus(tau-sigma_min(N)) to penalize unobservable modes. Optimize L_task+lambda_ctrl*L_ctrl. (3) The exact mathematical object is the PBH full-rank condition and the finite controllability matrix. Eigenvalues and singular values are estimated numerically. For large d, replace eigendecomposition with the controllability Gramian W_c=sum_{k=0}^{K-1} alpha^k gamma gamma^T (alpha^T)^k and penalize small eigenvalues of W_c using Lanczos iterations. (4) First experiment: train a width-64 SSM on Copy, Adding, and sequential-MNIST tasks, comparing vanilla, orthogonal-initialized, and regularized models at equal parameters and FLOPs. Track loss at fixed steps, long-range accuracy, gradient norms through 1k-4k tokens, smallest Gramian eigenvalue, and the number of near-unreachable modes. Success means faster loss descent or improved long-context accuracy without inference overhead.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: no_effect
- Verdict: Built a differentiable PBH controllability regularizer and a deterministic toy verification plus recurrent learning comparison. All three quantitative mechanism predictions were confirmed: PBH smallest singular value scaled as epsilon with observed slope 1.000000, the weak controllability-Gramian eigenvalue scaled as epsilon^2 with observed slope 2.000000, and delayed signal gain decayed with observed log slope -0.0833816, exactly matching log(0.92). In training, regularization improved the minimum Gramian eigenvalue from 0.000989 to 0.005962, about 6x stronger minimum controllability, while retaining nearly zero task loss; this demonstrates the mechanism but not a broad task-accuracy win.

### Mechanism check

- Verdict: Built a differentiable PBH controllability regularizer and a deterministic toy verification plus recurrent learning comparison. All three quantitative mechanism predictions were confirmed: PBH smallest singular value scaled as epsilon with observed slope 1.000000, the weak controllability-Gramian eigenvalue scaled as epsilon^2 with observed slope 2.000000, and delayed signal gain decayed with observed log slope -0.0833816, exactly matching log(0.92). In training, regularization improved the minimum Gramian eigenvalue from 0.000989 to 0.005962, about 6x stronger minimum controllability, while retaining nearly zero task loss; this demonstrates the mechanism but not a broad task-accuracy win.
- Confidence: 9/10
- Limitations: Only a small synthetic diagonal-transition recurrence was tested; alpha was fixed and only gamma was learned. Copy, adding, sequential-MNIST, non-diagonal/non-normal transitions, readout observability penalty, FLOPs, inference speed, and 1k-4k-token long-context sequences were not tested.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.3225
- Paired wins: 5/8
- Benchmark verdict: no measurable effect

## Artifacts

- [bench_experiment.py](https://synthcore.org/code/305/bench_experiment.py)
- [bench_report.json](https://synthcore.org/code/305/bench_report.json)
- [experiment.py](https://synthcore.org/code/305/experiment.py)
- [report.md](https://synthcore.org/code/305/report.md)
- [report_bench_2026-08-31T005310.md](https://synthcore.org/code/305/report_bench_2026-08-31T005310.md)
- [results.json](https://synthcore.org/code/305/results.json)
- [Download all files as ZIP](https://synthcore.org/download/305)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.