Empirical Minimal-Realisation Compression of Deep Neural Networks via Controllability-Observability Tests

What the math gives to ML

The paper reframes hidden-layer compression as empirical minimal-realisation of a depth-indexed dynamical system, separating directions that are excited by the data from directions that affect the output. Its transferable asset is the combination of reachability from hidden-state snapshots and observability from output Jacobians, followed by balanced truncation rather than weight magnitude or purely geometric low-rank approximation. This gives a layer-width selection rule tied to the task and data distribution, and can produce smaller inference graphs with little accuracy loss. The most direct ML transfer is an offline architecture-compression pass that computes balanced coordinates layer by layer, truncates low-energy state directions, and then briefly fine-tunes the resulting network.