# Эксперимент: Expansion-balanced MoE routing (#559)

{
  "worked": true,
  "confidence": 8,
  "verdict": "Built a differentiable expansion penalty, hard neighborhood diagnostics, and a matched 8-expert toy router benchmark in experiment.py. Uniform soft-count and finite-group predictions matched closed forms to below 5e-7 error, lambda scaling was exact, and the hinge activated at the predicted epsilon threshold. The regularizer reduced load standard deviation from 16.0 to 13.08, dropped-token fraction from 0.50 to 0.391, and subset violations from 1.0 to 0.292, while cross-entropy increased from 0.0013 to 0.182; this is a real load-balance signal but not an accuracy win.",
  "metrics": {
    "baseline": "CE 0.00128; max hard load 32; min hard load 0; load std 16.0; dropped fraction 0.500; violation fraction 1.000; mean hard neighborhood 3.896; soft expansion penalty 4.335",
    "idea": "CE 0.182; max hard load 32; min hard load 0; load std 13.077; dropped fraction 0.391; violation fraction 0.292; mean hard neighborhood 6.000; soft expansion penalty 0.714; formula errors <=4.7e-7; lambda scaling exact; predicted epsilon threshold 2.107 and observed hinge transition there"
  },
  "how_to_run": "/home/maxwelhelp/main/bin/python3 experiment.py",
  "files": [
    "experiment.py",
    "results.json"
  ],
  "limitations": "Only a synthetic 128-token, 8-expert top-1 router was tested with one random seed and a fixed subset schedule. No top-2 routing, capacity-factor sweep, FLOP-matched language-model training, C4 validation, multiple-seed statistics, or comparison against stronger modern load-balancing objectives was performed."
}