Эксперимент: Collision-Aware Subset Attention (#1253)

{ "worked": true, "confidence": 7, "verdict": "Built an exact local subset posterior router over 63 nonempty subsets for six-token neighborhoods, including pairwise compatibility products, normalized subset probabilities, and inclusion marginals. The math check passed: posterior mass summed to 1, marginal reconstruction error was 0, and boosting one pair compatibility increased its joint inclusion probability from 0.100 to 0.451. On the fixed synthetic collision task, the subset router outperformed the independent baseline in token F1 (0.987 vs 0.950) and reconstruction MSE (0.0055 vs 0.109), so the claimed signal was observed, although this is not yet evidence of a general transformer or FLOP-normalized win.", "metrics": { "baseline": "F1=0.9499, reconstruction MSE=0.10925", "idea": "F1=0.9874, reconstruction MSE=0.005536" }, "how_to_run": "python3 collision_subset_attention.py", "files": [ "collision_subset_attention.py", "results.json" ], "limitations": "Tested only one fixed seed on a small synthetic six-token collision dataset and 500 optimization steps. The comparison was equal-step rather than equal-FLOP; subset enumeration uses 63 subsets and therefore has substantially higher local compute than the independent router. No transformer/MoE integration, multiple-seed confidence interval, scaling study, or real-data evaluation was performed." }