Эксперимент: Descent-Certified LMO Sign Switching (#603)

{ "worked": true, "confidence": 8, "verdict": "Built exact SVD-polar post/pre candidates with independent noisy alignment gating and residual feedback. Three quantitative checks manifested: observed routing fractions exactly matched P(rho_hat >= tau), the linear loss change matched -eta< G,D > to numerical precision across all tested stepsizes, and routing agreement decreased monotonically from 1.00 to 0.765 as held-out gradient noise increased. In the matched noisy quadratic run, switching reduced uphill steps from 59 to 57 and improved best loss from 0.0421 to 0.0404 versus post-LMO, but its final loss 0.183 was worse than post-LMO’s 0.144; therefore this supports the safeguard mechanism, not a decisive overall optimization win.", "metrics": { "baseline": "Post-LMO SignMuon: final_loss=0.14447, best_loss=0.04207, uphill_steps=59/160; post alignment negative rate=0.346.", "idea": "Descent-certified switch at tau=0.05: final_loss=0.18320, best_loss=0.04038, uphill_steps=57/160, post-branch fraction=0.750; selected-direction negative true-alignment rate=0.0206 versus 0.346 for post-only. Threshold routing fractions at tau=[0,0.02,0.05,0.1] were [0.659,0.656,0.653,0.647], exactly matching empirical predicted fractions. Linear descent identity max absolute error across eta sweep=1.11e-16; routing agreement at held-out noise [0,0.25,0.55,1,2]=[1.000,0.974,0.937,0.882,0.765]." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 switch_experiment.py", "files": [ "switch_experiment.py", "results.json", "run_output.txt", "run_output2.txt" ], "limitations": "Only a 4x4 synthetic matrix and one matched-seed noisy quadratic comparison were tested; no CIFAR-10, nanoGPT, ResNet-18, multi-seed statistics, wall-clock/communication measurement, or GPU implementation was evaluated. The held-out gradient was simulated by additive noise rather than produced by an actual independent minibatch, and the threshold sweep’s branch-fraction prediction is definitional rather than an independent statistical model." }