Optimal Finite Interval Discrepancy via Binary Refinement

arXiv:2608.08431 2026 Architecture 1 ideas extracted · analyzed Sep 1, 2026

What the math gives to ML

The paper gives a sharp minimax law for progressively refining a conserved positive mass into binary pieces: no binary refinement schedule can keep the largest-to-smallest piece ratio below D_n = 2^{1-1/\lceil n/2\rceil} at every intermediate cardinality, and a lex-merge construction attains this bound. This transfers to progressive mixture-of-experts growth, hierarchical routers, and adaptive quantizers, where experts or bins are split while preserving their total routing probability or capacity. The useful asset is not merely balanced final weights, but an optimal guarantee on imbalance throughout the entire expansion trajectory. A practical adaptation is to split an expert's routing prior using a constrained search that keeps the current ratio close to the theorem's target, then initialize child router logits with matching probability offsets.

Ideas from this paper

Unverified 2026

Minimax-balanced progressive MoE splitting

Grow a mixture-of-experts layer by splitting one expert into two children while conserving its routing mass, and choose the split ratio to minimize the worst imbalance over all intermediate expert counts. Use the paper's sharp threshold as a hard design target: with n experts, some intermediate stage must have capacity ratio at least D_n = 2^{1-1/\lceil n/2\rceil}, so schedules substantially below this are impossible rather than merely difficult to discover. Initialize child router logits with…

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Optimal Finite Interval Discrepancy via Binary Refinement arXiv:2608.08431