Эксперимент: State-Routed Low-Rank MLP (#50)

{ "worked": false, "confidence": 8, "verdict": "Built a self-contained state-routed low-rank SwiGLU MVP with a diagonal stable SSM controller, soft routing, weighted rank-3 updates for all three projections, mathematical checks, and a seeded synthetic persistent-regime regression. The math check passed: spectral radius 0.9969<1, maximum update rank 3, softmax sum error 0, and homogeneous state norm contracted to 0.163. The idea reached lower validation MSE than baseline (0.9616 vs 0.9908, about 2.9% better), but required 24.44s versus 1.42s and 6284 versus 3456 parameters, failing the proposed <=10% throughput-overhead criterion.", "metrics": { "baseline": "validation MSE 0.99077; train MSE 1.01413; runtime 1.42s; parameters 3456", "idea": "validation MSE 0.96160; train MSE 0.97513; runtime 24.44s; parameters 6284; mean update Frobenius norm 1.9405; alpha entropy 0.4227; math spectral radius 0.99689" }, "how_to_run": "/home/maxwelhelp/main/bin/python3 experiment.py", "files": [ "experiment.py" ], "limitations": "Only one random seed and one small synthetic regression task were tested; no WikiText or language-model evaluation, Transformer integration, FLOP-matched comparison, multi-layer/shared-controller study, long-context extrapolation, or ablations were run. The routed implementation loops over sequence positions and materializes per-token matrices, so its measured speed is not representative of an optimized fused kernel." }