Finite-Horizon Hidden-State Observability Regularizer

Integration point: use a vanilla ReLU RNN or a recurrent state-space model with transition xₜ₊₁ = [W xₜ + U uₜ + b]₊. Choose C as a fixed coordinate-selection matrix, or learn a soft selection matrix and discretize it after training. During each training batch, unroll a prefix of length T while retaining x₀. Pseudocode: set P = I; store J_blocks = [C]; for each step compute z = W x + U u + b, D = diag(z > 0), update x = relu(z), update P = D W P, and append C P to J_blocks. Concatenate the blocks into J, form G = JᵀJ + εI, and optimize task_loss + λ times negative_logdet(G). For large n, estimate the smallest eigenvalue with Lanczos rather than constructing the full Jacobian. The paper supplies the mask-product Jacobian and the m ≥ n/2 lower-bound mechanism; the implementation estimates finite-batch Jacobians and conditioning empirically. First experiment: use an n = 16 ReLU RNN on permuted sequential MNIST or the copy-memory task, compare m = 2, 4, 7, 8, and 12, with and without the regularizer. Train an auxiliary decoder from y₀:T to reconstruct x₀. The quantitative prediction is a transition near m = 8 = n/2: below eight observed coordinates, increasing T should leave substantial hidden-state collisions, whereas at or above eight the smallest singular value and reconstruction accuracy should increase sharply. Measure whether the empirical transition lies within one observation node of n/2.