Эксперимент: Rank-One Delta Associative Memory (#485)

{ "worked": true, "confidence": 9, "verdict": "Built and numerically verified the rank-one delta memory update. The predicted scalar error multiplier, absolute-stability boundary at betalambda=2, convergence steps, and orthogonal-key independence matched observations to approximately 1e-15 or exactly on the tested grid, so the mechanism manifested. However, delta memory performed far worse than causal softmax attention on associative recall (MSE 1.1055 vs 8.03e-6), despite using 12x fewer state floats; no task-quality win was observed.", "metrics": { "baseline": "Causal softmax attention: MSE 8.03e-6, cosine 0.999999, 12,288 stored write floats", "idea": "Rank-one delta memory: MSE 1.1055, cosine 0.2875, 1,024 state floats; observed stability boundary betalambda=2.000; multiplier maximum error 2.22e-15; predicted and observed convergence steps matched at 88, 42, 19, and 19" }, "how_to_run": "/home/maxwelhelp/main/bin/python3 experiment.py", "files": [ "experiment.py", "results.json" ], "limitations": "Only a synthetic fixed-projection associative-recall task was tested; no learned projections, language model, WikiText/TinyStories, multi-head implementation, streaming segments, throughput, GPU memory, or matched-FLOP training comparison was evaluated. The attention baseline had direct access to all stored write keys and values, while delta memory compressed them into a fixed matrix." }