# Cone-Positive Ordered State-Space Layer

- ID: 1265
- Canonical URL: https://synthcore.org/idea/1265/cone-positive-ordered-state-space-layer
- API JSON: https://synthcore.org/api/idea/1265.json
- API Markdown: https://synthcore.org/api/idea/1265.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.22130](https://arxiv.org/abs/2607.22130)
- Category: dynamics
- Solves: stability, accuracy, generalization
- ML areas: rnn, ssm, training-dynamics
- Math tags: dynamical-systems, control-theory, geometry, linear-algebra, stochastic-processes
- Ratings: usefulness 8/10; difficulty 6/10; novelty 7/10

## Idea description

Replace an unconstrained recurrent transition by a unidirectional cooperative state-space update whose tangent dynamics preserve a positive cone. Add a penalty enforcing strict cone preservation and a spectral gap between the dominant ordered direction and transverse directions, so long sequences collapse toward a stable one-dimensional ordered manifold without eliminating nonlinear expressivity.

## Mathematical statement

For a continuous hidden-state system \(\dot h=f_\theta(h,u)\), let \(J(h,u)=\partial f_\theta/\partial h\) be its Jacobian and let \(C=\{v\in\mathbb R^N:v_i\ge0\}\) be the positive tangent cone. A sufficient cooperative condition is \(J_{ij}(h,u)\ge0\) for every off-diagonal entry, meaning that \(J\) is a Metzler matrix; then the variational flow \(\dot v=J(h(t),u(t))v\) preserves \(C\). Strict positivity is promoted by requiring a margin \(J_{ij}\ge\mu>0\) on the directed edges of an irreducible coupling graph. Differential positivity is approximated by requiring a dominant tangent growth rate \(\lambda_1\) separated from the next rate \(\lambda_2\), with transverse decay rate \(\lambda_1-\lambda_2>0\). Here \(h\) is the hidden state, \(u\) is the input, \(v\) is a tangent perturbation, \(\mu\) is the cone margin, and \(\lambda_i\) are finite-time Lyapunov exponents.

## Key formulas

- $$\dot h=f_\theta(h,u),\qquad \dot v=J(h,u)v,\qquad J(h,u)=\frac{\partial f_\theta}{\partial h}(h,u).$$
- $$J_{ij}(h,u)\ge\mu>0\quad(i\ne j\text{ on directed edges}),\qquad J\text{ Metzler}\ \Longrightarrow\ v(0)\in C\Rightarrow v(t)\in C.$$$
- $$\mathcal L_{\mathrm{cone}}=\frac{1}{B}\sum_{b=1}^{B}\sum_{(i,j)\in E}\left[\mu-J_{ij}(h_b,u_b)\right]_+^2+\gamma\left[\widehat\lambda_2-\widehat\lambda_1+\delta\right]_+^2,$$
- $$\|v_\perp(t)\|\lesssim e^{-\Delta t}\|v_\perp(0)\|,\qquad \Delta=\lambda_1-\lambda_2>0.$$

## Implementation notes

1. Integration point: implement a residual recurrent layer with hidden state \(h_t\in\mathbb R^N\), such as \(h_{t+1}=h_t+\eta[-a\odot h_t+\sigma(W_u u_t+W_hh_t+b)]\). Parameterize \(W_h\) as a directed lower-bidiagonal or lower-triangular matrix, with \((W_h)_{ij}=\operatorname{softplus}(q_{ij})\) on allowed edges \(i>j\); add small positive skip edges if an irreducible graph is needed. Keep diagonal damping \(a_i>0\). 2. Pseudocode: sample minibatch states and inputs; compute Jacobian-vector products \(Jv_r\) for random nonnegative tangent vectors; evaluate the cone-margin term above and add it to the task loss. Every \(K\) updates, propagate QR-normalized tangent vectors through the recurrent layer and estimate the top two finite-time Lyapunov exponents. Optimize task loss plus \(\beta\mathcal L_{\mathrm{cone}}\). At inference, inject small Gaussian perturbations into hidden states and measure order preservation. 3. Theoretical quantities are the Metzler condition and cone margin; finite-time exponents, projective contraction, and suitable \(\mu,\beta,\delta\) are estimated empirically. 4. First cheap experiment: compare a GRU, an unconstrained residual RNN, and this layer on sequential MNIST or adding and permuted-MNIST at sequence lengths 100, 500, and 1000. Measure cone violation rate \(\Pr[(Jv)_i<0]\) for \(v\ge0\), tangent-angle contraction, gradient norms, and accuracy. The quantitative prediction is that cone violations approach zero once the learned edge margin is positive, while transverse tangent norms decay as \(e^{-\Delta t}\); the fitted decay slope should agree with \(\widehat\lambda_1-\widehat\lambda_2\) within 20%. Reducing directed coupling below the irreducibility margin should remove the spectral gap and the one-dimensional long-horizon behavior.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built a cooperative residual RNN and analytic toy verification in experiment.py. For c>0, the Metzler flow was strictly positive in every tested case; the measured spectral gap matched N*c to numerical precision, projective contraction matched slope -N*c, and the explicit Euler cone-preservation transition occurred at eta*a=1. Cone positivity removed sampled Jacobian violations but reduced task accuracy in this tiny experiment, so no task-performance win was observed.

### Mechanism check

- Verdict: Built a cooperative residual RNN and analytic toy verification in experiment.py. For c>0, the Metzler flow was strictly positive in every tested case; the measured spectral gap matched N*c to numerical precision, projective contraction matched slope -N*c, and the explicit Euler cone-preservation transition occurred at eta*a=1. Cone positivity removed sampled Jacobian violations but reduced task accuracy in this tiny experiment, so no task-performance win was observed.
- Confidence: 9/10
- Limitations: Only a small synthetic sequence-sum task was tested, not sequential or permuted MNIST, GRU, long-horizon training, learned cone penalties, online Lyapunov estimation, irreducibility ablations, robustness perturbations, or repeated random seeds. The model uses direct positive parameterization rather than the proposed Jacobian penalty and spectral-gap regularizer.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/260/bench_report.json)
- [bench_run.py](https://synthcore.org/code/260/bench_run.py)
- [experiment.py](https://synthcore.org/code/260/experiment.py)
- [report.md](https://synthcore.org/code/260/report.md)
- [report_bench_2026-08-30T234044.md](https://synthcore.org/code/260/report_bench_2026-08-30T234044.md)
- [results.json](https://synthcore.org/code/260/results.json)
- [verification_summary.json](https://synthcore.org/code/260/verification_summary.json)
- [verify_report.py](https://synthcore.org/code/260/verify_report.py)
- [Download all files as ZIP](https://synthcore.org/download/260)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.