Statistical complexity from fluctuations in the information content

What the math gives to ML

The paper proposes the variance of information content as a statistical-complexity measure, C = Var[-log p], which vanishes for both deterministic and equiprobable distributions and is largest in intermediate structured regimes. For Boltzmann distributions it becomes C = β² Var(E), directly linking complexity to energy fluctuations and heat capacity, with a peak near continuous phase transitions. A transferable neural-network mechanism is to monitor or control the information-content variance of predictive distributions, attention weights, or mixture-of-experts routing probabilities. The main falsifiable prediction is a nonmonotonic complexity curve: collapse and uniform randomness both produce low C, while useful structured uncertainty produces an interior maximum.