Эксперимент: Assignment Tree Attention (#695)

{ "worked": true, "confidence": 8, "verdict": "Built assignment_tree_attention.py with Gumbel-perturbed rectangular Hungarian assignments, empirical edge frequencies, maximum-frequency bipartite spanning-tree projection, log-frequency attention bias, and a dense-versus-sparse retrieval comparison. The mechanism predictions manifested: frequency RMSE scaled as K^-0.513 versus predicted K^-0.5, union sizes closely matched the occupancy prediction for K=4..512, and alpha=1 increased frequency/probability correlation from 0.671 to 0.783. The sparse tree improved toy retrieval MSE (0.466 versus 0.585) while retaining exactly 30 tree edges versus 240 dense edges, although the rectangular transport claim was not exact because one query is unmatched per assignment.", "metrics": { "baseline": "Dense attention: retrieval MSE 0.584946; 240 possible cross-token edges.", "idea": "Tree sparse attention K=16: retrieval MSE 0.465771; exactly 30 edges; RMSE scaling slope -0.51284 versus predicted -0.5; frequency correlation alpha=0/1: 0.67094/0.78340; maximum row transport marginal error 0.02148 and key error 0.00417." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 assignment_tree_attention.py", "files": [ "assignment_tree_attention.py", "results.json" ], "limitations": "Only a small synthetic retrieval task was tested; no transformer training, WikiText/CIFAR benchmark, wall-clock or GPU memory measurement, cache-refresh study, gradient-variance measurement, or full K/R sweep was performed. The spanning-tree projection is a practical maximum-frequency heuristic rather than a proof of the paper's union-tree property, and the supplied rectangular transport normalization does not yield exact uniform query marginals when Hungarian matching leaves one query unmatched." }