Эксперимент: Shell-Wise Balanced MoE Routing (#522)

{ "worked": true, "confidence": 9, "verdict": "Built a deterministic top-1 shell-wise balanced router with quantile confidence shells and exact per-shell expert quotas. Across E={3,5,8} and K={2,4,7}, the observed maximum per-shell count spread was 1, matching the floor/ceil prediction; for divisible shell sizes, the Eq. 66 polynomial expert-marginal spread was exactly 0 over 99 excitation probabilities, while baseline spread was 17.5–20.3. In the synthetic imbalanced-routing comparison, CV fell from 1.095 to 0.000 and every shell reached zero CV, but 47.1% of tokens were reassigned with average router-score loss 0.509 per token; this demonstrates the mechanism, not an end-to-end perplexity win.", "metrics": { "baseline": "Aggregate expert loads [1627,1320,182,211,199,186,199,172], load CV=1.0948; Eq.66-grid maximum marginal spread=17.53-20.35 in divisible-shell sweeps.", "idea": "Aggregate loads [512,512,512,512,512,512,512,512], load CV=0.0000; per-shell count spread <=1 in all nondivisible sweeps; exact Eq.66-grid spread=0.0 for divisible shells; reassigned fraction=0.4712; router-score loss/token=0.5094." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 shell_moe_experiment.py", "files": [ "shell_moe_experiment.py", "results.json", "results.txt" ], "limitations": "Only a synthetic top-1 router was tested; top-2 routing, capacity overflow simulation, throughput/dispatch overhead, auxiliary-loss sensitivity, language-model training, perplexity, and GPU execution were not tested. The greedy assignment implementation does not yet implement the proposed top-r deficit-aware optimizer." }