△ Mechanism confirmed, baseline not beaten 2026

Cone-Positive Ordered State-Space Layer

Implementation & benchmark of arXiv:2607.22130 — Differential positivity and dynamical order in noisy oscillators under unidirectional coupling

Usefulness8/10

Difficulty6/10

Novelty7/10

Dynamics Stability Accuracy Generalization Rnn Ssm Training dynamics Dynamical systems Control theory Geometry Linear algebra Stochastic processes

Source paper: Differential positivity and dynamical order in noisy oscillators under unidirectional coupling arXiv:2607.22130 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace an unconstrained recurrent transition by a unidirectional cooperative state-space update whose tangent dynamics preserve a positive cone. Add a penalty enforcing strict cone preservation and a spectral gap between the dominant ordered direction and transverse directions, so long sequences collapse toward a stable one-dimensional ordered manifold without eliminating nonlinear expressivity.

Formulas

$$\dot h=f_\theta(h,u),\qquad \dot v=J(h,u)v,\qquad J(h,u)=\frac{\partial f_\theta}{\partial h}(h,u).$$

$$J_{ij}(h,u)\ge\mu>0\quad(i\ne j\text{ on directed edges}),\qquad J\text{ Metzler}\ \Longrightarrow\ v(0)\in C\Rightarrow v(t)\in C.$$$

$$\mathcal L_{\mathrm{cone}}=\frac{1}{B}\sum_{b=1}^{B}\sum_{(i,j)\in E}\left[\mu-J_{ij}(h_b,u_b)\right]_+^2+\gamma\left[\widehat\lambda_2-\widehat\lambda_1+\delta\right]_+^2,$$

$$\|v_\perp(t)\|\lesssim e^{-\Delta t}\|v_\perp(0)\|,\qquad \Delta=\lambda_1-\lambda_2>0.$$

Mathematical statement

For a continuous hidden-state system $\dot h=f_\theta(h,u)$, let $J(h,u)=\partial f_\theta/\partial h$ be its Jacobian and let $C=\{v\in\mathbb R^N:v_i\ge0\}$ be the positive tangent cone. A sufficient cooperative condition is $J_{ij}(h,u)\ge0$ for every off-diagonal entry, meaning that $J$ is a Metzler matrix; then the variational flow $\dot v=J(h(t),u(t))v$ preserves $C$. Strict positivity is promoted by requiring a margin $J_{ij}\ge\mu>0$ on the directed edges of an irreducible coupling graph. Differential positivity is approximated by requiring a dominant tangent growth rate $\lambda_1$ separated from the next rate $\lambda_2$, with transverse decay rate $\lambda_1-\lambda_2>0$. Here $h$ is the hidden state, $u$ is the input, $v$ is a tangent perturbation, $\mu$ is the cone margin, and $\lambda_i$ are finite-time Lyapunov exponents.

Implementation notes

1. Integration point: implement a residual recurrent layer with hidden state $h_t\in\mathbb R^N$, such as $h_{t+1}=h_t+\eta[-a\odot h_t+\sigma(W_u u_t+W_hh_t+b)]$. Parameterize $W_h$ as a directed lower-bidiagonal or lower-triangular matrix, with $(W_h)_{ij}=\operatorname{softplus}(q_{ij})$ on allowed edges $i>j$; add small positive skip edges if an irreducible graph is needed. Keep diagonal damping $a_i>0$. 2. Pseudocode: sample minibatch states and inputs; compute Jacobian-vector products $Jv_r$ for random nonnegative tangent vectors; evaluate the cone-margin term above and add it to the task loss. Every $K$ updates, propagate QR-normalized tangent vectors through the recurrent layer and estimate the top two finite-time Lyapunov exponents. Optimize task loss plus $\beta\mathcal L_{\mathrm{cone}}$. At inference, inject small Gaussian perturbations into hidden states and measure order preservation. 3. Theoretical quantities are the Metzler condition and cone margin; finite-time exponents, projective contraction, and suitable $\mu,\beta,\delta$ are estimated empirically. 4. First cheap experiment: compare a GRU, an unconstrained residual RNN, and this layer on sequential MNIST or adding and permuted-MNIST at sequence lengths 100, 500, and 1000. Measure cone violation rate $\Pr[(Jv)_i<0]$ for $v\ge0$, tangent-angle contraction, gradient norms, and accuracy. The quantitative prediction is that cone violations approach zero once the learned edge margin is positive, while transverse tangent norms decay as $e^{-\Delta t}$; the fitted decay slope should agree with $\widehat\lambda_1-\widehat\lambda_2$ within 20%. Reducing directed coupling below the irreducibility margin should remove the spectral gap and the one-dimensional long-horizon behavior.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built a cooperative residual RNN and analytic toy verification in experiment.py. For c>0, the Metzler flow was strictly positive in every tested case; the measured spectral gap matched N*c to numerical precision, projective contraction matched slope -N*c, and the explicit Euler cone-preservation transition occurred at eta*a=1. Cone positivity removed sampled Jacobian violations but reduced task accuracy in this tiny experiment, so no task-performance win was observed.

Agent confidence: 9/10

Baseline: Unconstrained residual RNN: test accuracy 0.8574; sampled off-diagonal Jacobian cone violation 0.5000.
Idea: Cone-positive residual RNN: test accuracy 0.6289; sampled off-diagonal Jacobian cone violation 0.0000. Toy sweeps matched gap predictions [0, 0.06, 0.18, 0.36, 0.60, 1.20] and projective slopes [0, -0.06, -0.18, -0.36, -0.60, -1.20]. Strict flow positivity matched c>0, and Euler preservation matched the eta*a<=1 boundary.

Limitations: Only a small synthetic sequence-sum task was tested, not sequential or permuted MNIST, GRU, long-horizon training, learned cone penalties, online Lyapunov estimation, irreducibility ablations, robustness perturbations, or repeated random seeds. The model uses direct positive parameterization rather than the proposed Jacobian penalty and spectral-gap regularizer.

How to run: python3 experiment.py && /home/maxwelhelp/main/bin/python3 verify_report.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.22130) Sep 2, 2026

Significantly WORSE than the tuned baseline (+16422.9%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0016
Idea mean: 0.2586
Effect (Δ): 0.257 (+16422.9%; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: n/a
Mechanism: Confirmed ✓
Practical effect: Hurts
Baseline tuning: swept over 3 configs

Limitations:

Only the built-in dynamics track was tested. The reduced-budget run used 10 epochs and 800/300 train/test examples; it did not test longer horizons, alternative widths, explicit learned cone penalties versus hard parameterization, irreducibility ablations, a GRU baseline, or multi-step rollout metrics.

How to run: python3 bench_run.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 30, 2026 latest Failed ✗

Agent confidence: 10/10

Implemented a matched cone-positive residual RNN on the structurally appropriate dynamics track using eight paired seeds, an equal learning-rate sweep, and trained-model mechanism measurements. The mechanism prediction was confirmed: cone violation was 0.0 for every trained idea model and the mean local dominant eigenvalue gap was 4.8856 with a positive gap on every seed. The idea was nevertheless significantly worse on independent dynamics rollout MSE: 0.258579 versus 0.001565, paired delta +0.257014, permutation p=0.0081.

Baseline: Dynamics rollout test MSE mean 0.0015649689, std 0.0003898933, 8 seeds; best sweep config lr=0.01, weight_decay=0.0.
Idea: Dynamics rollout test MSE mean 0.2585790250, std 0.0569780204, 8 seeds; best config lr=0.003, weight_decay=0.0. Mean cone violation 0.0, max cone violation 0.0, mean local eigenvalue gap 4.8856114, positive-gap fraction 1.0.

Benchmark result