Эксперимент: Residual-to-State Update Throttle (#820)

{ "worked": true, "confidence": 9, "verdict": "Built a deterministic residual-to-state throttle for masked linear regression, with exact gradient scaling and scalar stability checks. The predicted gain boundary a<1 occurred at residual norm κ√q with 0.06% relative sweep error; in the throttled regime, residual_norm·a/√q matched κ to about 4e-9. Plain scalar GD diverged for η>2 while throttled updates remained bounded in all tested cases. The mini-experiment showed the idea improved final error over SGD at η=0.24 (0.209 vs 0.353) and prevented instability, but was worse at η=0.08 because its mean gain was only 0.066, so this is a mechanism win and promising stability signal, not a universal optimization win.", "metrics": { "baseline": "Masked SGD: final parameter error 0.247, 0.311, 0.353 at learning rates 0.08, 0.16, 0.24; scalar η=2.5 final magnitude 4.07e17 and η=3.0 final magnitude 1.27e30.", "idea": "Throttle: final parameter error 0.977, 0.397, 0.209 at learning rates 0.08, 0.16, 0.24; scalar η=2.5 final magnitude 0.464 and η=3.0 final magnitude 0.400. Mean gains were 0.066-0.082 in the masked regression runs." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 sweeps.py && /home/maxwelhelp/main/bin/python3 experiment.py", "files": [ "experiment.py", "sweeps.py", "results.json", "sweep_results.json" ], "limitations": "The main mini-experiment used exact q for a linear model rather than Hutchinson Jacobian estimates or a two-layer MLP. It did not test large neural networks, wall-clock/FLOP overhead, realistic observability preconditioners B, noisy/nonlinear residual geometry, or longer recurrent-excitation trajectories. Adam was included only as a small comparison and was not hyperparameter-tuned." }