When trajectory-based bounds fail: information thermodynamics under noisy feedback

What the math gives to ML

The paper’s transferable mechanism is a noise-dependent ranking of information measures in feedback systems: trajectory-dependent quantities such as transfer entropy degrade rapidly when observations are noisy and control histories are temporally correlated, whereas instantaneous mutual-information measures remain comparatively robust. This suggests replacing fixed trajectory-information regularizers in partially observed neural dynamical systems with an adaptive estimator that switches toward instantaneous dependence when measurement noise crosses a measurable threshold. The key falsifiable prediction is a crossover in which the estimated trajectory-information signal falls faster with observation noise than the instantaneous-information signal. This is most relevant to RNNs, state-space models, world models, and learned feedback controllers trained from noisy sensor streams.