Эксперимент: Curvature-Band SAM Direction (#1183)

{ "worked": false, "confidence": 9, "verdict": "Built a Hessian-polynomial curvature-band SAM prototype with two HVPs per polynomial iteration, plus an exact finite-dimensional spectral sanity check. The math claim was clearly observed: for rho=3, eigenvalues 2 and 4 were selected and the unwanted-component ratio fell from 1.19 to 1.77e-11, consistent with the polynomial gap. However, on the tiny two-moons MLP, curvature-band SAM matched SAM/SGD accuracy (0.844) but had higher random-perturbation loss increase (1.63e-4 vs SAM 1.18e-4), so there is no observed practical win.", "metrics": { "baseline": "SGD: test loss 0.3561, accuracy 0.8438, random perturbation loss increase 1.20e-4", "idea": "Curvature-band SAM: test loss 0.3549, accuracy 0.8438, random perturbation loss increase 1.63e-4, mean selected-direction Rayleigh quotient 0.419; spectral exact ratio 1.19 -> 1.77e-11" }, "how_to_run": "/home/maxwelhelp/main/bin/python3 experiment.py", "files": [ "experiment.py", "results.json" ], "limitations": "Only one tiny two-moons dataset, one seed, and a small MLP were tested; CIFAR-10, multiple seeds, top-Hessian evaluation, persistent directions, and systematic rho/eta/K/gamma tuning were not tested. The rho estimate used only two random HVP probes, and the reported curvature comparison uses a small random-perturbation proxy." }