Renormalization group and long-range conditional mutual information in hierarchical models
arXiv:2609.02141
2026
Architecture
2 ideas extracted · analyzed Sep 3, 2026
What the math gives to ML
The paper supplies a useful design principle for hierarchical neural representations: coarse-graining should retain variables that mediate long-range dependence, while testing whether discarded within-block information remains predictive of distant features. Its majority-spin example is a concrete sufficient-statistic construction, and its counterexample shows that a locally plausible pooling rule can destroy global conditional information even when pairwise mutual information is small. The most promising neural translation is a learned hierarchical pooling layer trained with a conditional-information preservation objective, optionally paired with a locally reversible decoder to reduce activation memory.
Ideas from this paper
✗ Mechanism failed
2026
Replace ordinary token merging or graph pooling with a learned block map whose output preserves information about a remote target conditioned on the surrounding coarse representation. The paper's majority-spin counterexample gives a concrete failure mode: two microscopic configurations mapped to the same pooled token can imply different predictions for distant variables.
Useful7/10
Difficulty5/10
Novelty6/10
Unverified
2026
Construct each hierarchy transition as a coarse-graining map together with a local decoder that recovers the retained local representation. This creates a reversible-information path for features needed by later computation while still allowing high-frequency or locally irrelevant components to be discarded.
Useful6/10
Difficulty6/10
Novelty5/10