Community structure of the pseudofractal web

arXiv:2607.03010 2026 Architecture 1 ideas extracted · analyzed Aug 30, 2026

What the math gives to ML

The paper provides an exactly solvable example showing that ordinary community evidence can mistake heavy-tailed degree variation for structure, while degree-corrected evidence recovers a nested hierarchy. The transferable asset is not the pseudofractal graph itself, but the model-selection principle: compare a partition against a degree-preserving null and recursively refine only when corrected evidence increases. This suggests a hierarchical router or graph pooling module whose assignments are driven by residual interaction structure rather than raw token frequency, node degree, or popularity. The strongest first test is a degree-corrected hierarchical MoE router on synthetic power-law data and a small language model, measuring whether it reduces expert collapse and improves quality at fixed communication cost.

Ideas from this paper

Mechanism works 2026

Degree-Corrected Hierarchical Router

Replace a flat MoE or graph-pooling assignment with recursive partitions selected by interaction evidence after removing each item’s expected degree effect. Tokens, nodes, or examples that are frequently active for purely popularity-related reasons should not automatically form an expert or cluster. Recursion stops when a candidate split has nonpositive degree-corrected evidence, producing an adaptive hierarchy rather than a fixed number of equally sized groups.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Community structure of the pseudofractal web arXiv:2607.03010