On the Number of Observation Nodes in Recurrent Neural Networks with Linear Threshold and ReLU Functions
The paper gives a transferable finite-horizon observability mechanism for recurrent networks. Its strongest result is that general real-valued ReLU dynamics can lose state information through activation masks, yielding a global lower bound…