Эксперимент: Centered Heavy-Tail Clipping Optimizer (#1347)

{ "worked": true, "confidence": 8, "verdict": "Built a reproducible NumPy MVP implementing coordinate-median centered residual clipping and verified its core behavior. The clipping operator preserved direction exactly and bounded residuals by tau; the measured tail-bias log-log slope was -0.506 versus the predicted 1-alpha=-0.5. In the quadratic toy experiment, centered clipping reached mean final loss 0.00122 under heavy tails versus 0.0563 for global clipping and 1.59 for unclipped SGD, while also improving the Gaussian control.", "metrics": { "baseline": "Heavy-tail final loss: SGD 1.5905, global clipping 0.05627; Gaussian global clipping 0.001683. Heavy-tail global clipping had mean p99 update norm 0.150.", "idea": "Centered clipping: heavy-tail final loss 0.001221 over 12 seeds, Gaussian final loss 0.000715; mean p99 update norms 1.329 and 1.329 respectively; zero divergences. Tail-bias slope -0.506 versus predicted -0.5; operator max residual 0.73 for tau=0.73 and minimum direction cosine 1.0." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 experiment.py", "files": [ "experiment.py", "results.json" ], "limitations": "This is a synthetic 20-dimensional quadratic oracle experiment, not a 2-layer MLP or CIFAR-10 test. The center is a coordinate-wise median and tau is fixed rather than adaptively estimated; no Adam compatibility, momentum, mirror-map variant, wall-clock or FLOP accounting, or broader tail, tau, and learning-rate sweep was tested. Global clipping uses the same small tau and therefore clips the common gradient direction, so the comparison demonstrates the proposed mechanism but is not a fully tuned optimizer benchmark." }