Эксперимент: Laplace-Heterogeneous MoE Routing (#693)

{ "worked": true, "confidence": 9, "verdict": "Built a reusable NumPy Laplace-heterogeneous MoE router with EMA pressure, finite-mixture availability, effective hazard, and top-k selection. The mechanism checks were confirmed: m'(A)=-Var_A(lambda) had max error 2.65e-8, q'/q=-m had error 4.12e-7, and the observed 1% asymptotic crossover pressure 7.65 closely matched the predicted 7.60. In the deterministic routing stress test, the mixture reduced load CV from 2.397 to 0.180 and overflow from 0.777 to 0.014 versus baseline, but this does not establish language-model quality improvement.", "metrics": { "baseline": "Toy top-1 routing: mean load CV 2.3968, overflow 0.7773, entropy 1.4998; final loads [469,32,3,1,0,2,4,1].", "idea": "Laplace mixture routing: mean load CV 0.1802, overflow 0.0141, entropy 1.9113; final loads [36,78,71,68,68,64,63,64]. Math errors: derivative identity 2.65e-8 and hazard identity 4.12e-7; crossover observed/predicted 7.65/7.60." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 experiment.py", "files": [ "experiment.py", "laplace_moe.py", "results.json" ], "limitations": "Only a deterministic NumPy toy routing stress test was run; no trained Transformer, language-model loss, perplexity, CUDA benchmark, learned mixture parameters, top-2 dispatch capacity implementation, or equal-FLOP training comparison was tested." }