Эксперимент: Co-Prime Virtual-Aperture Attention (#1219)

{ "worked": true, "confidence": 8, "verdict": "Built a causal co-prime virtual-aperture attention MVP with M1=3 and M2=4, plus sparse and dense baselines. The math check reproduced 6 physical offset families, 12 unique virtual offsets, exact graph reachability, and fewer virtual offsets for the non-coprime (2,4) control. On a seeded lag-17 copy task, CPA reached 99.83% accuracy and loss 0.0201 at the same 7 attention edges per token as the sparse baseline, which reached 4.80% accuracy and loss 2.9467; this is a clear toy-task signal for long-range coverage, but not a speed win.", "metrics": { "baseline": "single sparse: loss 2.9467, accuracy 0.0480, 3.195 s; dense: loss 2.7531, accuracy 0.1445, 0.788 s", "idea": "CPA: loss 0.0201, accuracy 0.9983, 3.406 s; 7 attention edges/token, 12 virtual offsets, 50% of virtual offsets >8" }, "how_to_run": "/home/maxwelhelp/main/bin/python3 coprime_attention_experiment.py", "files": [ "coprime_attention_experiment.py", "results.json", "run_output.txt", "run_output_lag17.txt" ], "limitations": "This is a tiny synthetic one-layer experiment with sequence length 24, one coprime pair, one non-coprime math control, and 180 optimization steps. It does not test language-model perplexity, multi-layer transformers, realistic data, equal FLOPs including projection costs, optimized sparse kernels, activation memory, or generalization beyond the fixed lag-17 task; measured wall-clock time was not improved." }