Optimal Finite Interval Discrepancy via Binary Refinement
arXiv:2608.08431
2026
Architecture
1 ideas extracted · analyzed Sep 1, 2026
What the math gives to ML
The paper gives a sharp minimax law for progressively refining a conserved positive mass into binary pieces: no binary refinement schedule can keep the largest-to-smallest piece ratio below D_n = 2^{1-1/\lceil n/2\rceil} at every intermediate cardinality, and a lex-merge construction attains this bound. This transfers to progressive mixture-of-experts growth, hierarchical routers, and adaptive quantizers, where experts or bins are split while preserving their total routing probability or capacity. The useful asset is not merely balanced final weights, but an optimal guarantee on imbalance throughout the entire expansion trajectory. A practical adaptation is to split an expert's routing prior using a constrained search that keeps the current ratio close to the theorem's target, then initialize child router logits with matching probability offsets.
Ideas from this paper
Unverified
2026
Grow a mixture-of-experts layer by splitting one expert into two children while conserving its routing mass, and choose the split ratio to minimize the worst imbalance over all intermediate expert counts. Use the paper's sharp threshold as a hard design target: with n experts, some intermediate stage must have capacity ratio at least D_n = 2^{1-1/\lceil n/2\rceil}, so schedules substantially below this are impossible rather than merely difficult to discover. Initialize child router logits with…
Useful6/10
Difficulty5/10
Novelty8/10