Эксперимент: Krylov Block-Cubic Optimizer (#7)

{ "worked": false, "confidence": 9, "verdict": "Built a CPU MVP of the Krylov block-cubic optimizer using Hessian-vector products, reorthogonalized Lanczos bases, reduced cubic subproblem solves, adaptive sigma, and an actual-loss monotonicity guard. The exact quadratic sanity check succeeded with stationarity residual 1.796e-10 and Lanczos orthogonality error 1.906e-15. On the fixed synthetic MLP regression task, cubic optimization reached loss 0.024539 versus Adam's 0.009479 in similar wall time, with only 18/35 steps accepted, so the promised optimization win was not observed.", "metrics": { "baseline": "Adam, 80 steps: final/best MSE 0.009479, wall time 2.762 s, no HVPs", "idea": "Krylov cubic, 35 steps: final/best MSE 0.024539, wall time 2.938 s, 18/35 accepted, 70 HVPs" }, "how_to_run": "/home/maxwelhelp/main/bin/python3 experiment.py", "files": [ "experiment.py", "results.txt" ], "limitations": "Only one small synthetic full-batch regression task, one fixed seed, one full parameter block, and k=6 Lanczos were tested. No MNIST, stochastic training, block partitioning, wall-clock/FLOP-normalized sweeps, or comparison with L-BFGS/Newton-CG was performed." }