{
 "artifacts": null,
 "category": "architecture",
 "description": "Constrain recurrent preactivations to remain nonnegative so that ReLU acts as the identity along realized trajectories. The hidden dynamics then admit a classical linear observability matrix, allowing principled hidden-coordinate selection and conditioning control instead of relying on potentially destructive activation masks.",
 "formulas_latex": [
  "xₜ₊₁ = [A xₜ + B uₜ + b]₊,    A xₜ + B uₜ + b ≥ 0 componentwise ⇒ xₜ₊₁ = A xₜ + B uₜ + b",
  "δxₜ₊₁ = A δxₜ,    Q_T = [C; CA; CA²; …; CAᵀ]",
  "rank(Q_T) = n,    σ_min(Q_T) \u003e 0",
  "L_pos = Σₜ ||[−A xₜ − B uₜ − b]₊||₂²,    L_cond = −log det(Q_Tᵀ Q_T + εI)"
 ],
 "id": 2844,
 "implementation": "Integration point: replace the hidden transition of a small recurrent state-space model with xₜ₊₁ = relu(A xₜ + B uₜ + b), and expose m selected hidden coordinates through yₜ = C xₜ or an auxiliary memory decoder. Add the positivity and observability losses to the task loss. Pseudocode: unroll the sequence and compute zₜ = A xₜ + B uₜ + b; accumulate the positivity penalty ||relu(−zₜ)||²; update xₜ₊₁ = relu(zₜ). Separately construct Q = [C; CA; …; CAᵀ] for a chosen horizon T, calculate QᵀQ + εI, and add its negative log determinant to the objective. Optionally constrain A with spectral normalization if bounded recurrent dynamics are also desired. The paper supplies the exact condition under which ReLU observability reduces to linear observability; the positivity rate and singular values are measured empirically. First experiment: use n = 16 on the copy-memory and adding-problem datasets, compare unconstrained ReLU, positive-penalty ReLU, and tanh RNNs with m = 4 observed coordinates. Evaluate horizons T = 1, 2, 4, 8, and 16. The prediction is that when the negative-preactivation fraction falls below 1%, reconstruction quality follows the linear criterion: rank deficiency produces σ_min(Q_T) ≈ 0 and high reconstruction error, while full rank yields decreasing error as T increases until the smallest singular value saturates. Report whether observed rank transitions match Q_T within numerical tolerance.",
 "math_summary": "Consider xₜ₊₁ = [A xₜ + B uₜ + b]₊, where xₜ ∈ Rⁿ is the state, uₜ is the input, A and B are matrices, and b is a bias. If A xₜ + B uₜ + b is componentwise nonnegative for all relevant states and inputs, then ReLU is inactive and the dynamics become affine linear. Perturbations satisfy δxₜ₊₁ = A δxₜ. With observations yₜ = C xₜ, the finite-horizon observability matrix is Q_T = [C; CA; …; CAᵀ]. The initial state is observable when rank(Q_T) = n, and robustness is measured by the smallest singular value of Q_T or the condition number of Q_TᵀQ_T. A trainable positivity penalty is the squared norm of the negative preactivation, which encourages the regime in which the paper's exact reduction to linear observability applies.",
 "math_tags": [
  "control-theory",
  "dynamical-systems",
  "linear-algebra",
  "spectral-theory"
 ],
 "ml_areas": [
  "rnn",
  "ssm",
  "memory",
  "regularization"
 ],
 "paper": {
  "arxiv_id": "2608.29650",
  "arxiv_url": "https://arxiv.org/abs/2608.29650",
  "summary_what_math_gives_to_ml": "The paper gives a transferable finite-horizon observability mechanism for recurrent networks. Its strongest result is that general real-valued ReLU dynamics can lose state information through activation masks, yielding a global lower bound of m ≥ n/2 observed hidden coordinates, while nonnegative ReLU trajectories reduce exactly to linear-system observability. A practical neural-network transfer is to regularize the finite-horizon observation Jacobian so that hidden-state information is preserved, and to use positivity constraints when a classical linear observability certificate is desirable.",
  "title": "On the Number of Observation Nodes in Recurrent Neural Networks with Linear Threshold and ReLU Functions",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 6,
  "usefulness": 7
 },
 "solves": [
  "stability",
  "accuracy",
  "generalization"
 ],
 "title": "Positive-Regime Observable ReLU State Space",
 "url": "https://synthcore.org/idea/2844/positive-regime-observable-relu-state-space",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}