△ Mechanism confirmed, baseline not beaten 2026

Van der Corput progressive expert scheduler

Implementation & benchmark of arXiv:2608.02559 — House-monotone multi-level apportionment has logarithmic quota discrepancy

Usefulness7/10

Difficulty5/10

Novelty7/10

Architecture Stability Sample efficiency Accuracy Moe routing Transformer Training dynamics Combinatorics Number theory Approximation theory

Source paper: House-monotone multi-level apportionment has logarithmic quota discrepancy arXiv:2608.02559 ⓘ · analyzed Aug 31, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace random or greedy one-expert-at-a-time activation with a deterministic binary van der Corput sequence. At each training or inference step, the schedule chooses an expert whose cumulative usage remains close to its proportional target, while recursively balancing nested expert groups rather than only balancing individual experts.

Formulas

$$v_2(m)=\sum_{j\ge 0} b_j2^{-j-1},\qquad m=\sum_{j\ge 0}b_j2^j,\quad b_j\in\{0,1\}.$$

$$x_m=\left\lfloor D\,v_2(m)\right\rfloor,\qquad A_S(n)=\sum_{t=1}^{n}\mathbf 1\{x_t\in S\},\qquad E_S(n)=A_S(n)-\frac{n|S|}{D}.$$

$$\max_{S\in\mathcal C}\max_{n\ge 1}|E_S(n)|\le \frac{\log D}{3\log 2}+1,$$

$$\Delta_{\mathrm{interval}}\le \Delta_{\mathrm{grid}}+\frac12.$$

Mathematical statement

For D equally entitled leaves, let x_t in {0,...,D-1} be the expert selected at step t and let A_S(n)=sum_{t=1}^n 1{x_t in S} be the number of selections from a leaf set S in the first n steps. The ideal proportional count is n|S|/D, and the prefix discrepancy is E_S(n)=A_S(n)-n|S|/D. The paper's constructive schedule is the binary van der Corput ordering: write m=sum_{j>=0} b_j 2^j with bits b_j in {0,1}, define v_2(m)=sum_{j>=0} b_j 2^{-j-1}, and choose x_m=floor(D v_2(m)) for D a power of two. For the comb hierarchy, the resulting worst nested-prefix error is bounded by log(D)/(3 log 2)+1. A midpoint embedding gives general interval discrepancy at most the grid-aligned discrepancy plus 1/2. The transferable property is deterministic proportional coverage at every prefix, rather than merely good average frequencies.

Implementation notes

Integrate the scheduler into a top-1 MoE router or another progressive activation mechanism where one of D equally sized experts is selected per token or microbatch. For global assignment index m, compute the binary radical inverse v2(m), set the scheduled expert c=floor(D*v2(m)) for D a power of two, and either force c for a small calibration fraction or use c as the first candidate before selecting among the learned router logits. A hybrid implementation is: compute router logits l; reserve a fraction rho of assignments for the schedule; on scheduled assignments choose expert c; otherwise choose argmax(l), subject to capacity. Maintain A_S(n) for every node S in a fixed binary tree over experts and calculate E_S(n)=A_S(n)-n|S|/D after each batch. The exact mathematical inputs are v2, the one-hot schedule, and the logarithmic discrepancy budget B(D)=log(D)/(3 log 2)+1. Router logits, rho, tree layout, and capacity factor are empirical choices. First test a 4- or 8-expert Switch-Transformer-style MLP on CIFAR-100 or a small language-model corpus against random routing and standard capacity-factor routing at identical FLOPs. Record maximum nested-group discrepancy, expert-load variance, dropped-token rate, and validation loss. Success means fewer overflowed tokens and lower worst-prefix imbalance without worse perplexity or classification accuracy.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built a readable binary van der Corput scheduler and numerical verification experiment in files vdc_moe_experiment.py and results.json. Across D=2..64, endpoint error at n=D was exactly 0, maximum nested discrepancy was 0.5..0.984 (always below the stated B(D)=1.33..3.0), and VDC leaf prefix error stayed below 1. In the synthetic capacity test with D=8 and 128 assignments, VDC had zero drops and zero load variance at capacity factor 1.0, versus 9.3% drops for random routing and 61.0% for greedy routing; this demonstrates the scheduling mechanism, not end-to-end model accuracy.

Agent confidence: 9/10

Baseline: Random routing: at D=8, mean final leaf imbalance 3.275 and mean max nested-prefix discrepancy 4.484; capacity factor 1.0 averaged 9.30% drops and load variance 4.16. Greedy routing: 60.98% drops and load variance 17.94 at capacity factor 1.0.
Idea: VDC: at D=8, max leaf/nested prefix discrepancy 0.875, endpoint error at n=D exactly 0, and capacity factor 1.0 averaged 0% drops and 0 load variance. Across D=2..64, max nested discrepancy remained below B(D) and max leaf error remained below 1.

Limitations: The experiment is a scheduler/capacity simulation rather than a trained Switch-Transformer or CIFAR/language-model benchmark. It does not test router-logit integration, learned expert specialization, FLOPs, validation loss, classification accuracy, perplexity, GPU behavior, or the midpoint interval-discrepancy claim; the random baseline uses a reduced trial count for runtime.

How to run: python3 vdc_moe_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.02559) Sep 2, 2026

Ahead of the baseline (−3.4%) but not statistically significant (p=0.0805, wins 6 / 8 paired seeds).

Benchmark: Expert_balanced_regression
Model: top1_moe_shared
Paired seeds: 8
Baseline mean: 0.2129
Idea mean: 0.2056
Effect (Δ): -0.0073 (−3.4%; negative = idea better)
Wins: 6 / 8 paired seeds
p-value: 0.0805 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±3.8%
Mechanism: Confirmed ✓
Practical effect: Inconclusive
Baseline tuning: swept over 6 configs

Limitations:

The custom task is a small synthetic regression rather than a standard Switch Transformer or large-scale language/vision benchmark. Capacity overflow/drop-rate behavior was not implemented in the neural training loop, nested-group discrepancy beyond leaf prefixes was not measured, and the baseline routing sweep used random routing rather than a learned greedy router.

How to run: python3 registered_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 31, 2026 latest Failed ✗

Agent confidence: 9/10

The registered custom track expert_balanced_regression was accepted by the bench smoke test and evaluated with identical end-to-end top-1 MoE systems, paired seeds, baseline sweep, and make_report(). VDC improved mean test MSE from 0.2129017226 to 0.2056338880, but the paired permutation test was not significant (p=0.08055), so the required benchmark verdict is no significant win. The trained-model mechanism signature was confirmed: observed maximum leaf prefix discrepancy was 0.875 versus the prediction <=1.0, with final load variance 0.0.

Baseline: Registered random-routing top-1 MoE, tuned lr=0.01, epochs=12: mean test MSE 0.2129017226; per-seed [0.2303046435, 0.1982028186, 0.1861430258, 0.2568784952, 0.2142492831, 0.1867903024, 0.2180261165, 0.2126190960].
Idea: Registered VDC-routing top-1 MoE, lr=0.01, rho=1.0, epochs=12: mean test MSE 0.2056338880; paired delta -0.0072678346; 6/8 wins; p=0.08055; observed prefix discrepancy 0.875 and final load variance 0.0.

Benchmark result