# Triangular Hierarchical Neural State-Space Layer

- ID: 596
- Canonical URL: https://synthcore.org/idea/596/triangular-hierarchical-neural-state-space-layer
- API JSON: https://synthcore.org/api/idea/596.json
- API Markdown: https://synthcore.org/api/idea/596.md
- Verification status: beats_baseline
- Source: [arXiv:2607.08827](https://arxiv.org/abs/2607.08827)
- Category: dynamics
- Solves: stability, speedup, scalability
- ML areas: ssm, rnn, training-dynamics
- Math tags: dynamical-systems, control-theory, linear-algebra, spectral-theory
- Ratings: usefulness 7/10; difficulty 5/10; novelty 6/10

## Idea description

Replace an unconstrained recurrent transition with a hierarchy of features whose generator is triangular: degree-ell features depend only on degree-ell and lower-degree features. This transfers the paper's closure mechanism for even-Majorana monomials into a neural state-space model, preserving nonlinear feature interactions while making the spectrum and long-time transients directly controllable.

## Mathematical statement

The paper defines Heisenberg evolution by \(\frac{d}{dt}O=\mathcal{L}^{\dagger}[O]=\mathrm{i}[H,O]+\mathcal{D}^{\dagger}[O]\), where \(O\) is an observable, \(H\) is the Hamiltonian, and \(\mathcal{D}^{\dagger}\) is the adjoint dissipator. For linear Majorana jumps, reduced even-Majorana monomials form a closed triangular hierarchy: ordering features by monomial degree gives \(\dot z=Az\), with \(A_{ij}=0\) whenever feature \(i\) depends on a higher-level feature \(j\). Adapt this as a neural feature hierarchy \(z=(z_1,\ldots,z_K)\), where \(z_k\) contains interactions of order or scale \(k\), and parameterize \(A\) as block lower triangular with diagonal blocks \(A_{kk}\). The exact spectral stability condition is \(\max_k\alpha(A_{kk})<0\), where \(\alpha(B)=\max\{\operatorname{Re}\lambda:\lambda\in\operatorname{spec}(B)\}\). If the largest Jordan block associated with eigenvalue real part \(-\alpha<0\) has size \(r\), then perturbations satisfy a transient bound of the form \(\|e^{At}\|\le C(1+t^{r-1})e^{-\alpha t}\); hierarchy depth controls the polynomial transient and diagonal damping controls asymptotic decay.

## Key formulas

- $$\frac{d}{dt}O=\mathcal{L}^{\dagger}[O]=\mathrm{i}[H,O]+\mathcal{D}^{\dagger}[O].$$
- $$\dot z=Az,\qquad A=\begin{pmatrix}A_{11}&0&0\\A_{21}&A_{22}&0\\A_{31}&A_{32}&A_{33}\end{pmatrix},$$
- $$\alpha(A)=\max_k\alpha(A_{kk})<0\quad\Longrightarrow\quad z(t)=e^{At}z(0)\text{ decays asymptotically},$$
- $$\|e^{At}\|\le C(1+t^{r-1})e^{-\alpha t},\qquad \alpha>0.$$

## Implementation notes

1. Integration point: implement this as the recurrent transition in a small state-space model, neural ODE, or residual RNN. Split the hidden state into K groups, such as local features, pairwise features, and higher-order features. Use an unconstrained encoder to produce input-dependent forcing, but constrain the hidden-to-hidden matrix to block lower-triangular form. Parameterize each diagonal block as \(A_{kk}=S_k-\operatorname{softplus}(d_k)I\), where \(S_k\) is skew-symmetric and \(d_k>0\), guaranteeing negative diagonal-block spectral abscissa. 2. Pseudocode: compute \(u=\operatorname{Encoder}(x_t)\); set \(z'_1=A_{11}z_1+B_1u\); for \(k=2,\ldots,K\), set \(z'_k=A_{kk}z_k+\sum_{j<k}A_{kj}z_j+B_ku\); integrate with a matrix exponential for fixed A or RK4 for input-dependent A; predict from \([z_1;\ldots;z_K]\). During training add a penalty on estimated positive-real-part eigenvalues if the diagonal blocks are not analytically constrained. 3. Computed from the paper: the closure and triangular topology. Estimated empirically: block eigenvalues, the constant C, and effective transient degree r. 4. First cheap experiment: compare this layer with a GRU and a diagonal SSM on Adding, Copying, and sequential-MNIST tasks using identical parameter counts. Sweep the largest diagonal decay parameter and integration step. The falsifiable prediction is that perturbations follow an initial polynomial-times-exponential envelope \(t^{r-1}e^{-\alpha t}\), while long-run dynamics remain stable whenever \(\max_k\alpha(A_{kk})<0\). Gradually change one diagonal block from decay \(-\alpha\) to growth \(+\alpha\); the observed stability transition should occur within 20% of zero spectral abscissa. Increasing hierarchy depth from K=2 to K=4 should increase transient amplification approximately polynomially without changing the asymptotic exponential decay rate.

## Verification

- Status: beats_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: helps
- Verdict: Built a triangular hierarchical state-space layer with analytically damped diagonal blocks, skew-symmetric within-block dynamics, and lower-triangular cross-level couplings. Mechanism checks confirmed the predicted stability transition at growth rate 0, exact polynomial-times-exponential Jordan responses, peak-time errors below 0.0004, corrected decay slope -0.7, and Lambda^(K-1) peak scaling with observed/predicted ratios of 0.999999995. The triangular model slightly beat the small Adding-task baselines, but that comparison was not parameter-matched; the strongest result is confirmation of the stability and transient mechanism.

### Mechanism check

- Verdict: Built a triangular hierarchical state-space layer with analytically damped diagonal blocks, skew-symmetric within-block dynamics, and lower-triangular cross-level couplings. Mechanism checks confirmed the predicted stability transition at growth rate 0, exact polynomial-times-exponential Jordan responses, peak-time errors below 0.0004, corrected decay slope -0.7, and Lambda^(K-1) peak scaling with observed/predicted ratios of 0.999999995. The triangular model slightly beat the small Adding-task baselines, but that comparison was not parameter-matched; the strongest result is confirmation of the stability and transient mechanism.
- Confidence: 9/10
- Limitations: The task comparison used unequal parameter counts and one short Adding-task setup. Copying, sequential-MNIST, FLOP or speed measurements, multiple random seeds, and learned matrix-exponential integration were not tested. The neural implementation uses explicit Euler integration, and the CUDA attempt fell back to CPU after a cuDNN host-allocation error.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 8/8
- Benchmark verdict: idea better (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/270/bench_report.json)
- [mechanism_results.json](https://synthcore.org/code/270/mechanism_results.json)
- [report.md](https://synthcore.org/code/270/report.md)
- [report_bench_2026-08-31T000902.md](https://synthcore.org/code/270/report_bench_2026-08-31T000902.md)
- [results.json](https://synthcore.org/code/270/results.json)
- [run_bench.py](https://synthcore.org/code/270/run_bench.py)
- [run_experiment.py](https://synthcore.org/code/270/run_experiment.py)
- [verify_mechanism.py](https://synthcore.org/code/270/verify_mechanism.py)
- [Download all files as ZIP](https://synthcore.org/download/270)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.