# Differentially Positive Recurrent Core

- ID: 2314
- Canonical URL: https://synthcore.org/idea/2314/differentially-positive-recurrent-core
- API JSON: https://synthcore.org/api/idea/2314.json
- API Markdown: https://synthcore.org/api/idea/2314.md
- Verification status: unverified
- Source: [arXiv:2608.14980](https://arxiv.org/abs/2608.14980)
- Category: dynamics
- Solves: stability, generalization
- ML areas: rnn, ssm, training-dynamics, regularization
- Math tags: dynamical-systems, control-theory, geometry, differential-geometry
- Ratings: usefulness 6/10; difficulty 5/10; novelty 6/10

## Idea description

Constrain the Jacobian of a recurrent or state-space transition to preserve a prescribed cone of admissible hidden-state perturbations. This imports differential positivity into neural dynamics and makes long-run hidden trajectories order-preserving rather than allowing arbitrary sign-changing perturbation growth.

## Mathematical statement

Let the hidden dynamics be $h_{t+1}=F_\theta(h_t,x_t)$, with Jacobian $J_t=\partial F_\theta/\partial h(h_t,x_t)$. A cone field is a family of closed, convex, pointed cones $K(h)\subset T_h\mathbb{R}^d$; in the simplest implementation use the fixed positive orthant $K=\mathbb{R}^d_+$. Differential positivity requires $J_tK\subseteq K$, equivalently $J_{t,ij}\geq0$ for every entry when $K=\mathbb{R}^d_+$. For a state-dependent cone, use a matrix $C(h)$ whose columns generate $K(h)$ and require $J_tC(h_t)=C(h_{t+1})A_t$ for some nonnegative matrix $A_t\geq0$. The paper's recurrent-dynamics consequence is that every connected component of the Birkhoff center, the set of recurrent points, and the support of any invariant measure is either strongly ordered or unordered under the cone-induced relation. In a neural implementation, the cone condition is enforced softly by penalizing negative entries of $J_t$ or by factorizing positive transition weights.

## Key formulas

- $$K(h)=\mathbb{R}^d_+,\qquad J_tK\subseteq K\ \Longleftrightarrow\ (J_t)_{ij}\geq0\quad\forall i,j,$$
- $$h_{t+1}=F_\theta(h_t,x_t),\qquad J_t=\frac{\partial F_\theta}{\partial h}(h_t,x_t),$$
- $$J_tC(h_t)=C(h_{t+1})A_t,\qquad A_t\geq0,$$
- $$\mathcal{L}_{\mathrm{cone}}(\theta)=\frac{1}{Td^2}\sum_{t=0}^{T-1}\sum_{i,j}\operatorname{softplus}(-J_{t,ij}/\tau),\qquad J_t\text{ cone-preserving when }\mathcal{L}_{\mathrm{cone}}=0.$$

## Implementation notes

1. Exact integration point: replace the recurrent matrix in a small GRU, vanilla RNN, or diagonal-plus-low-rank SSM by a nonnegative transition matrix. For a first implementation use $h_{t+1}=\phi(W_hh_t+W_xx_t+b)$ with $W_h=\operatorname{softplus}(V_h)$ and a monotone activation such as softplus; this guarantees positive-orthant cone preservation when no sign-changing skip terms are added. A less restrictive version keeps unconstrained weights and adds the Jacobian penalty. 2. Pseudocode: initialize $V_h$; at each sequence step compute $h_{t+1}$, retain $J_t=\partial h_{t+1}/\partial h_t$ using forward-mode Jacobian-vector products or exact Jacobians for a small model; set $L=L_{task}+\lambda L_{cone}$; backpropagate and update $\theta$. For larger models estimate violations with $m$ random vectors $v_k\geq0$: penalize $\operatorname{ReLU}(-J_tv_k)$ instead of materializing $J_t$. 3. Computed from the paper: cone preservation and the order-based recurrent interpretation. Estimated empirically: Jacobian entries, cone-violation rate, Lyapunov exponents, and whether recurrent hidden states form ordered or unordered clusters. Define violation rate as the fraction of sampled $(t,k)$ for which any component of $J_tv_k$ is negative. 4. First cheap experiment: train a one-layer RNN on adding, copy-memory, and sequential-MNIST tasks, comparing an unconstrained RNN, a positive-transition RNN, and the soft-penalty model at matched parameter counts. Sweep transition gain $\alpha$ in $W_h=\alpha\operatorname{softplus}(V_h)$ and sequence length. The quantitative prediction is that the hard-positive model has zero cone violations, while the soft model's violation rate decreases monotonically with $\lambda$; ordered perturbation pairs should preserve componentwise order over time. A falsifiable long-horizon signature is that positive models avoid the abrupt error increase seen when the unconstrained model's largest estimated Jacobian-product Lyapunov exponent becomes positive.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.