Эксперимент: Spiderweb Hierarchical Attention (#630)

{ "worked": true, "confidence": 8, "verdict": "Built a readable NumPy Spiderweb hierarchy with dyadic pooling, bounded same-level neighborhoods, attention updates, and downward broadcast. The math checks confirmed all three targeted predictions: equal-height hyperbolic distance decreased monotonically across levels, the inverse-metric transition level matched the discrete argmin with 96% exact agreement and mean error 0.04 levels, and normalized work remained bounded while dense work grew by 29.1x over the tested range. The impulse test showed global communication (far-token influence 0.00638 versus zero for local attention), while estimated work was 3,835 versus 65,536 dense pair scores; however, the unoptimized NumPy implementation was slower wall-clock than dense attention, so the demonstrated win is structural work/memory and long-range reach rather than runtime.", "metrics": { "baseline": "Dense: 65,536 pair scores, 8.97e-5 s/call, far impulse influence 0.003906. Local: 1,280 pair scores, 0.00854 s/call, far impulse influence 0.", "idea": "Spiderweb: estimated 3,835 operations (1,531 horizontal + 2,304 broadcast), 0.1151 s/call in unoptimized NumPy, far impulse influence 0.00637899; O(n log n) normalized work ranged 2.37 to 1.64." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 spiderweb_experiment.py && /home/maxwelhelp/main/bin/python3 run_benchmark.py && /home/maxwelhelp/main/bin/python3 impulse_check.py", "files": [ "spiderweb_experiment.py", "run_benchmark.py", "impulse_check.py", "benchmark_results.json", "impulse_results.json" ], "limitations": "No learned Transformer, language-model perplexity, WikiText/long-range-copy training, matched parameter/FLOP optimization, GPU kernel, peak memory measurement, or hyperbolic-distance routing ablation was tested. The runtime comparison is not favorable because the reference implementation uses slow Python/NumPy loops and is not an optimized sparse kernel." }