Positive-Regime Observable ReLU State Space

Integration point: replace the hidden transition of a small recurrent state-space model with xₜ₊₁ = relu(A xₜ + B uₜ + b), and expose m selected hidden coordinates through yₜ = C xₜ or an auxiliary memory decoder. Add the positivity and observability losses to the task loss. Pseudocode: unroll the sequence and compute zₜ = A xₜ + B uₜ + b; accumulate the positivity penalty ||relu(−zₜ)||²; update xₜ₊₁ = relu(zₜ). Separately construct Q = [C; CA; …; CAᵀ] for a chosen horizon T, calculate QᵀQ + εI, and add its negative log determinant to the objective. Optionally constrain A with spectral normalization if bounded recurrent dynamics are also desired. The paper supplies the exact condition under which ReLU observability reduces to linear observability; the positivity rate and singular values are measured empirically. First experiment: use n = 16 on the copy-memory and adding-problem datasets, compare unconstrained ReLU, positive-penalty ReLU, and tanh RNNs with m = 4 observed coordinates. Evaluate horizons T = 1, 2, 4, 8, and 16. The prediction is that when the negative-preactivation fraction falls below 1%, reconstruction quality follows the linear criterion: rank deficiency produces σ_min(Q_T) ≈ 0 and high reconstruction error, while full rank yields decreasing error as T increases until the smallest singular value saturates. Report whether observed rank transitions match Q_T within numerical tolerance.