Эксперимент: Effective-resistance natural-gradient routing (#370)

{ "worked": true, "confidence": 9, "verdict": "Built an exact fixed-m external-field router with DP subset sampling, exact covariance/Fisher computation, pseudoinverse natural-gradient routing, and resistance-based pairwise trust scaling. The mechanism manifested: across 65 imbalanced configurations, the maximum normalized resistance ratio was 0.999996, the covariance lower-bound violation was only numerical noise (minimum eigenvalue -2.0e-14), and trust-cap sweeps matched predicted min(rho, raw_step) to machine precision. In the toy optimization, natural-trust reached loss 1.03e-7 versus 1.47e-3 for vanilla after 80 steps, though this is not evidence of a production MoE win.", "metrics": { "baseline": "Toy exact-router optimization: final loss 0.001474, step-10 loss 0.05923, step-40 loss 0.005067, final load CV 0.6995", "idea": "Toy exact-router natural-gradient plus rho=0.35 cap: final loss 1.034e-7, step-10 loss 0.001425, step-40 loss 8.850e-6, final load CV 0.7296; resistance ratio max 0.999996; covariance-bound minimum eigenvalue -2.0e-14; trust-cap observed/predicted equal for rho={0.01,0.03,0.1,0.3,1.0}; DP sampling max marginal error 0.00119 over 200000 samples" }, "how_to_run": "/home/maxwelhelp/main/bin/python3 experiment.py > results.json && /home/maxwelhelp/main/bin/python3 verify_router.py", "files": [ "experiment.py", "results.json", "router.py", "verify_router.py" ], "limitations": "No CIFAR, language modeling, transformer, expert compute, communication-cost, or minibatch-load experiment was tested. Exact covariance uses subset enumeration and is practical only for small expert counts; the DP sampler is implemented, but DP marginal computation for large d and a CUDA training integration were not implemented." }