Single-Node Observable Leaky-RNN

Use this as a sparse RNN or SSM architecture. First choose a directed graph with $n$ hidden nodes and maximum indegree $K$; designate one sensor node $s$, preferably a node reached by many directed paths. Parameterize every existing recurrent edge as $a_{ij}=\operatorname{softplus}(u_{ij})+\epsilon$ to match the positive-weight setting, retain a trainable bias, and use Leaky-ReLU with fixed $\alpha\in\{0.01,0.1,0.5\}$. During each training example, record only $y_t=x_s(t)$ and feed the length-$T+1$ scalar sequence to a small decoder $D_\phi$ that predicts the full initial hidden state $x_0$. Add $L_{obs}=\|D_\phi(y_{0:T})-\operatorname{stopgrad}(x_0)\|_2^2$ to the task loss. The update procedure is: initialize graph and sensor; run $x_{t+1}=F_\theta(x_t)$; store $y_t=x_s(t)$; compute $\hat{x}_0=D_\phi(y_{0:T})$; backpropagate task loss plus $\lambda L_{obs}$; periodically estimate pairwise observation separations and rewire the sensor if the margin is too small. The paper supplies the injectivity mechanism and graph-based observability construction; the decoder, separation margin, and conditioning estimate are empirical additions. For a cheap falsifiable experiment, compare a one-sensor graph against dense and random-sensor GRUs on permuted-MNIST or sequential CIFAR-10, using equal hidden size and parameter count. Generate pairs of nearby initial states, measure $\delta_{\{s\},T}$ and decoder error as $T$ increases, and compare Leaky-ReLU to ordinary ReLU. The prediction is a horizon transition: after the longest relevant graph path reaches the sensor, reconstruction error should fall sharply and $\delta_{\{s\},T}$ should become positive or substantially larger. ReLU should show more collisions and dead-coordinate failures. Adding sensors should reduce the minimum horizon needed to reach a fixed reconstruction tolerance.