# Эксперимент: Projection-Regularized Gradient Updates (#590)

{
  "worked": true,
  "confidence": 9,
  "verdict": "Built an exact small-model ProjectionRegularizedSGD optimizer and a deterministic quadratic verification harness. The measured stability thresholds matched the analytical thresholds to about 1e-9; increasing lambda and rho monotonically increased the threshold, while alpha=0 gave exactly zero unsupported-direction metric and increasing alpha increased it. In the noisy quadratic comparison at eta=0.055, the projected method reached mean final loss 0.0232 versus 1.511 for the nonprojected regularized baseline, but this is secondary because that baseline is not plain SGD.",
  "metrics": {
    "baseline": "Noisy quadratic, nonprojected regularized metric (alpha=1): mean final loss 1.5106 +/- 1.1749 over 8 seeds",
    "idea": "Noisy quadratic, projection-regularized metric (lambda=0.1, rho=0.15, alpha=0.08): mean final loss 0.02316 +/- 0.00415 over 8 seeds",
    "ridge_thresholds": "Predicted/observed eta_c: lambda=.01 0.16944022/0.16944022; .03 0.17300721/0.17300721; .1 0.18552857/0.18552857; .3 0.22161991/0.22161991; 1.0 0.35161534/0.35161534",
    "rho_thresholds": "Predicted/observed eta_c: rho=.01 0.17920330/0.17920330; .03 0.18010930/0.18010930; .1 0.18327402/0.18327402; .3 0.19226259/0.19226259; 1.0 0.22311154/0.22311154",
    "alpha_support": "Unsupported q: alpha=0: 0.0; .02: 0.2; .1: 1.0; .3: 3.0; 1.0: 10.0"
  },
  "how_to_run": "/home/maxwelhelp/main/bin/python3 projection_experiment.py",
  "files": [
    "projection_experiment.py",
    "projection_optimizer.py",
    "results.json"
  ],
  "limitations": "Only a 2D deterministic/noisy quadratic and a small PyTorch smoke test were evaluated; no CIFAR, recurrent, transformer, or realistic small-data fine-tuning experiment was run. The optimizer uses explicit p-by-p matrices and is limited to small models; no wall-clock, FLOP, Hessian-vector-product, clipping, AdamW, or plain-SGD comparison was tested."
}