On the Observability and Controllability of Leaky-ReLU Networks

What the math gives to ML

The paper provides a constructive observability and controllability theory for sparse recurrent networks whose node updates are affine maps followed by an injective Leaky-ReLU. Its key transferable mechanism is that a small set of hidden nodes can encode the entire latent state over a finite time window, provided the directed dependency graph propagates information from every state coordinate to the measured nodes; bijectivity additionally enables finite-horizon state steering through selected actuated nodes. A practical neural-network transfer is an observability-aware sparse RNN or state-space model with graph-selected readout nodes, trained with a finite-window reconstruction loss and monitored by an empirical distinguishability margin.