Эксперимент: Bi-Maxwell Muon (#1039)

{ "worked": true, "confidence": 9, "verdict": "Built a reusable two-buffer Bi-Maxwell Muon implementation with fast/slow EMA relaxation and Newton-Schulz semi-orthogonalization. The mechanism manifested: discrete half-life crossings matched predictions for all tested betas, impulse log-decay slopes matched log(beta) to numerical precision, and frequency-response gains agreed within 0.02%-8.6%. In the tiny regression comparison, Bi-Maxwell did not improve optimization loss over Muon or AdamW, so this is mechanism validation rather than evidence of a practical training win.", "metrics": { "baseline": "Muon: final loss 0.001597, step-100 loss 1.532585, mean gradient/update cosine 0.5865, 0.141 s; AdamW: final loss 0.001178, step-100 loss 0.001263, cosine 0.3449, 0.068 s.", "idea": "Bi-Maxwell (beta_fast=0.90, beta_slow=0.99, w=0.5): final loss 0.004175, step-100 loss 1.533368, cosine 0.5502, 0.137 s. Bi-Maxwell (0.95,0.995,0.5): final loss 0.023674, step-100 loss 1.537552, cosine 0.5018, 0.152 s. Predicted-versus-observed half-life integer crossings matched 5/5; impulse slopes matched to numerical precision; frequency-gain relative errors were 8.6%, 2.35%, 0.22%, and 0.024%." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 verify.py", "files": [ "bi_maxwell_muon.py", "verify.py", "verification.json", "experiment.py", "results.json" ], "limitations": "The optimizer test was a small full-batch linear regression, not a decoder-only transformer or C4/WikiText experiment. It did not test distributed aggregation, mixed precision, GPU behavior, equal-FLOP transformer training, extensive hyperparameter tuning, validation perplexity, or multiple random seeds." }