Эксперимент: PSD-plus-low-rank curvature optimizer (#1208)

{ "worked": false, "confidence": 9, "verdict": "Built a PSD-plus-low-rank curvature optimizer with EMA diagonal PSD curvature, minibatch Hessian-vector products, rank-2 residual estimation, signed correction, and step clipping. The quadratic sanity check reproduced the predicted stability boundary: spectral radius was 0.90 at 0.95ηc and 1.10 at 1.05ηc, with one negative residual eigenvalue while the full Hessian remained PSD. On the fixed-seed digits MLP, however, the proposed method underperformed SGD substantially, so no neural-training win was observed.", "metrics": { "baseline": "SGD, lr=0.12, 90 steps: final loss 1.5128, accuracy 56.22%, diverged=false", "idea": "PSD-low-rank, nominal lr=0.12 with internal clipping, rank=2 and refresh every 15 steps: final loss 2.9691, minimum loss 1.6280, accuracy 15.78%, diverged=false", "toy": "ηc=1.4569; spectral radius 0.90 below the boundary and 1.10 above it; one negative residual eigenvalue" }, "how_to_run": "/home/maxwelhelp/main/bin/python3 psd_lowrank_experiment.py", "files": [ "psd_lowrank_experiment.py", "results.json" ], "limitations": "Only one small digits dataset, one nominal learning rate, one rank, one seed, and 90 update steps were tested; no learning-rate sweep or FLOP-normalized comparison was performed. The residual curvature was estimated with tiny randomized subspace iteration on minibatch Hessian-vector products, and the neural method was not tuned against SGD." }