✗ Failed on benchmark 2026

Controllability-Regularized State-Space Layer

Implementation & benchmark of arXiv:2607.10956 — Based maps to Lagrangian Grassmannians, Quivers, and Bott Periodicity

Usefulness7/10

Difficulty5/10

Novelty6/10

Dynamics Stability Accuracy Generalization Ssm Rnn Training dynamics Regularization Control theory Linear algebra Dynamical systems Optimization

Source paper: Based maps to Lagrangian Grassmannians, Quivers, and Bott Periodicity arXiv:2607.10956 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace an unconstrained latent transition in an SSM or recurrent block by quiver data (alpha,gamma), where alpha evolves the latent state and gamma injects token or feature inputs. Add a differentiable penalty that detects eigenmodes of alpha not reached from gamma, preventing dead latent directions and improving long-context signal propagation. The paper’s exact open condition becomes a practical regularizer rather than a hard architectural constraint.

Formulas

$$\operatorname{Im}(\alpha-\lambda\cdot\operatorname{Id}_{A})+\operatorname{Im}(\gamma)=A\quad\text{for all }\lambda\in\overline{{\mathbbm{k}}}.$$

$$Z=\left\{(\alpha,j)\in V_{\mp,n,d}:\exists\lambda\in\overline{{\mathbbm{k}}},v\in A\text{ such that }j(v)=0\text{ and }\alpha v=\lambda v\right\}.$$

$$\mathcal{C}(\alpha,\gamma)=\begin{bmatrix}\gamma&\alpha\gamma&\alpha^{2}\gamma&\cdots&\alpha^{d-1}\gamma\end{bmatrix},\qquad d=\dim A.$$

$$\mathcal{L}_{\mathrm{ctrl}}=\sum_{\lambda\in\operatorname{spec}(\alpha)}\operatorname{softplus}\!\left(\tau-\sigma_{\min}\!\left([\alpha-\lambda I_A\ \ \gamma]\right)\right)+\eta\,\operatorname{softplus}\!\left(\tau-\sigma_{\min}\!\left(\begin{bmatrix}\alpha-\lambda I_A\\ j\end{bmatrix}\right)\right).$$

Mathematical statement

The paper uses vector spaces A and W, an endomorphism alpha in Hom(A,A), and an input map gamma:W to A. Its key condition is Im(alpha-lambda I_A)+Im(gamma)=A for every lambda in the algebraic closure of the base field. Here I_A is the identity on the latent state space, lambda ranges over eigenvalues of alpha, and Im denotes image. Equivalently, the concatenated matrix [alpha-lambda I_A, gamma] has full row rank dim(A); this is the PBH controllability criterion. The paper also identifies the failure locus Z={ (alpha,j): there exist lambda and nonzero v with j(v)=0 and alpha v=lambda v }, meaning that an eigenmode is invisible to the output map j:A to U. For a neural SSM, gamma is the input injection and j is the readout. Penalize small singular values of these PBH matrices at eigenvalues of alpha. An eigenvalue detached from gradients may be used when computing the penalty for numerical stability.

Implementation notes

(1) Integration point: modify the transition and input-injection matrices of a discrete-time SSM or recurrent layer. For token features u_t in R^m and latent state h_t in R^d, use h_{t+1}=rho(alpha h_t+gamma u_t) and y_t=j h_t. Keep the rest of the neural block unchanged so the comparison isolates the regularizer. (2) Pseudocode: initialize alpha in R^{dxd}, gamma in R^{dxm}, and j in R^{rxd}; at each optimization step compute the task loss from the recurrence. Every K steps, calculate lam=eigvals(alpha.detach()). For each lam, form M=concat(alpha-lam*I,gamma) and compute its smallest singular value s. Add softplus(tau-s) to L_ctrl. Optionally form N=vertical_stack(alpha-lam*I,j) and add eta*softplus(tau-sigma_min(N)) to penalize unobservable modes. Optimize L_task+lambda_ctrl*L_ctrl. (3) The exact mathematical object is the PBH full-rank condition and the finite controllability matrix. Eigenvalues and singular values are estimated numerically. For large d, replace eigendecomposition with the controllability Gramian W_c=sum_{k=0}^{K-1} alpha^k gamma gamma^T (alpha^T)^k and penalize small eigenvalues of W_c using Lanczos iterations. (4) First experiment: train a width-64 SSM on Copy, Adding, and sequential-MNIST tasks, comparing vanilla, orthogonal-initialized, and regularized models at equal parameters and FLOPs. Track loss at fixed steps, long-range accuracy, gradient norms through 1k-4k tokens, smallest Gramian eigenvalue, and the number of near-unreachable modes. Success means faster loss descent or improved long-context accuracy without inference overhead.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not tested

Stage 2 · Practical benchmark result: No measurable effect

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built a differentiable PBH controllability regularizer and a deterministic toy verification plus recurrent learning comparison. All three quantitative mechanism predictions were confirmed: PBH smallest singular value scaled as epsilon with observed slope 1.000000, the weak controllability-Gramian eigenvalue scaled as epsilon^2 with observed slope 2.000000, and delayed signal gain decayed with observed log slope -0.0833816, exactly matching log(0.92). In training, regularization improved the minimum Gramian eigenvalue from 0.000989 to 0.005962, about 6x stronger minimum controllability, while retaining nearly zero task loss; this demonstrates the mechanism but not a broad task-accuracy win.

Agent confidence: 9/10

Baseline: Final task MSE 0.0; step-50 MSE 9.07e-4; step-200 MSE 2.15e-9; minimum PBH sigma 0.05583; minimum Gramian eigenvalue 9.89e-4.
Idea: Final task MSE 5.34e-8; step-50 MSE 8.11e-4; step-200 MSE 7.63e-8; minimum PBH sigma 0.05598; minimum Gramian eigenvalue 5.96e-3; approximately 6x stronger minimum controllability.

Limitations: Only a small synthetic diagonal-transition recurrence was tested; alpha was fixed and only gamma was learned. Copy, adding, sequential-MNIST, non-diagonal/non-normal transitions, readout observability penalty, FLOPs, inference speed, and 1k-4k-token long-context sequences were not tested.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.10956) Aug 31, 2026

Ahead of the baseline (−0.1%) but not statistically significant (p=0.3225, wins 5 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0458
Idea mean: 0.0457
Effect (Δ): -0 (−0.1%; negative = idea better)
Wins: 5 / 8 paired seeds
p-value: 0.3225 (permutation test, 20 000 shuffles)
Smallest detectable effect: n/a
Mechanism: n/a
Practical effect: No effect
Baseline tuning: swept over 0 configs

Limitations:

Reduced to 400 training and 150 test examples and 4 epochs after the unreduced run exceeded 2400 seconds. The PBH penalty was evaluated intermittently and applied to the GRU candidate gate only; observability regularization, longer rollouts, and larger hidden dimensions were not tested.

How to run: python3 bench_experiment.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 31, 2026 latest Failed ✗

Agent confidence: 9/10

Built the controllability-regularized rnn_small system on the actuated-pendulum dynamics benchmark, using the canonical baseline training path and a local auxiliary-loss loop for the intervention. Across 8 paired seeds, the idea's MSE delta was -4.28e-05 with permutation p=0.3225, which is not a significant win. The mechanism signature was not confirmed because input sensitivity increased but minimum PBH singular value decreased.

Baseline: Best lr=0.006, weight_decay=0.0001; test MSE mean 0.0457723, std 0.0140136.
Idea: Best reg_weight=0.02 at the same lr and weight_decay; test MSE mean 0.0457295, std 0.0139204; paired delta -4.2798e-05, 5/8 wins, p=0.3225.
Mechanism signature: map[baseline:map[min_pbh_sigma:0.01258535 observed_sensitivity:0.00376744] confirmed:false idea:map[min_pbh_sigma:0.01217466 observed_sensitivity:0.0043237] predicted_direction:Both minimum PBH sigma and input-output sensitivity should increase.]

Benchmark result