Эксперимент: Second-order SCAFFOLD bias compensation (#1204)

{ "worked": true, "confidence": 7, "verdict": "Built a scalar full-participation SCAFFOLD simulation with EMA estimates of curvature, third derivative, and noise variance, plus output and gradient bias correction. The finite-difference check recovered f''=1 and f'''=0.8, and the estimated bias formula closely matched the exact coefficient (-6.42e-5 vs -6.47e-5 in the sanity check). Across five seeds at H=8 and gamma=0.028, correction reduced aggregate absolute stationary mean bias from 4.49e-4 to 1.58e-4 at N=8 and from 3.81e-4 to 8.46e-5 at N=64; variance was essentially unchanged. The promised effect is observed, although gamma-scaling evidence was noisy.", "metrics": { "baseline": "At gamma=0.028,H=8: N=8 mean bias -4.49e-4, seed-averaged |bias| 8.02e-4, cross-seed SD 7.79e-4; N=64 mean bias -3.81e-4, seed-averaged |bias| 4.09e-4, SD 3.43e-4.", "idea": "Output correction: N=8 aggregate reported bias +1.58e-4, seed-averaged |bias| 6.75e-4; N=64 +8.46e-5, seed-averaged |bias| 3.31e-4. Gradient correction was nearly identical: +1.60e-4 and +8.50e-5, with unchanged variance." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 experiment.py && /home/maxwelhelp/main/bin/python3 aggregate.py", "files": [ "experiment.py", "aggregate.py", "results.json", "run_output.txt", "aggregate.txt" ], "limitations": "The MVP uses a scalar toy objective, synthetic Gaussian rather than bounded noise, and a stabilizing quartic term outside the local derivative specification. It does not test heterogeneous clients, minibatches, transient-loss curves, wall-clock/FLOP cost, or a broad statistically powered gamma-scaling study; the implemented estimator also uses the mathematically appropriate second difference of f' rather than the dimensionally inconsistent third-difference stencil stated in the implementation plan." }