# Эксперимент: Key-Selective Delta Momentum (#903)

{
  "worked": true,
  "confidence": 8,
  "verdict": "Built a normalized sequential Key-Selective Delta Momentum buffer and a reproducible toy regression experiment. The mechanism predictions were confirmed: residual contraction factors matched the exact formula (including the stability boundary alpha≈2 for ||x||²=5), orthogonal-key interference was 0, and old-direction retention was 1.0 for delta versus the EMA prediction/observation 0.9^10=0.348678. In the mini-experiment, best delta alpha=0.003 reached loss 0.002243 at step 150 versus momentum 0.005413 and final weight MSE 5.39e-6 versus 6.34e-6, but this is not evidence of a general speed/FLOP win because the delta method performs 64 sequential matrix updates per batch.",
  "metrics": {
    "baseline": "SGD-style EMA momentum: loss@150=0.005413, loss@300=0.002241, final weight MSE=6.34e-6",
    "idea": "Best normalized delta alpha=0.003: loss@150=0.002243, loss@300=0.002236, final weight MSE=5.39e-6; alpha sweep [0.001,0.003,0.01,0.03,0.1,0.3] showed degradation at extremes"
  },
  "how_to_run": "/home/maxwelhelp/main/bin/python3 experiment.py",
  "files": [
    "experiment.py",
    "results.json",
    "run.log",
    "alpha_sweep.json"
  ],
  "limitations": "Only a 2D synthetic linear regression was tested; no MLP, CNN/ResNet, CIFAR-10, AdamW, wall-clock/FLOP accounting, vectorized minibatch implementation, or systematic learning-rate retuning was evaluated. The toy uses sequential updates and a deliberately shifted input distribution, so the observed accuracy/loss signal may not transfer to neural networks."
}