Эксперимент: Garding Geometric-Mean Load Balancer (#1415)

{ "worked": false, "confidence": 8, "verdict": "Built a log-space geometric-mean load-balancing utility and a fixed-seed toy MoE router experiment. The mathematical claim was verified: 1,000 Jensen tests showed no concavity violation, and the Hessian had maximum eigenvalue approximately -1.1e-16. The structured GM variant reduced soft-load CV to 1.351 versus 1.617 for Switch-style balancing, but increased task loss to 0.277 versus 0.176 and did not improve hard routing imbalance or capacity-1 drop rate. The literal symmetric c=1 version is constant on the normalized-load simplex and supplies no balancing gradient there, so this MVP does not demonstrate a practical win.", "metrics": { "baseline": "Switch proxy: task loss 0.176159, soft-load CV 1.617491, hard-load CV 1.732051, capacity-1 drop rate 0.75; entropy: task loss 0.169341, soft-load CV 1.650789, hard-load CV 1.732051, drop rate 0.75.", "idea": "Structured GM: task loss 0.277067, soft-load CV 1.351020, hard-load CV 1.732051, capacity-1 drop rate 0.75; symmetric GM: task loss 0.165555, soft-load CV 1.682564, hard-load CV 1.732051, drop rate 0.75. Concavity check: mean Jensen gap -0.02108, maximum violation -4.61e-05, maximum Hessian eigenvalue -1.12e-16." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 experiment.py", "files": [ "experiment.py", "results.json" ], "limitations": "This is a small differentiable per-token router toy rather than a language model with learned hidden states, top-k dispatch, EMA loads, or realistic capacity handling. Coefficients and auxiliary weights were not swept, and hard argmax routing remained concentrated on the two task-preferred experts, so the structured utility's soft-load improvement did not translate to dispatch-level improvement." }