Эксперимент: q-Ary Influence Overlap Regularizer (#1199)

{ "worked": false, "confidence": 8, "verdict": "Built an exhaustive Lemma 3.1 verifier and a CUDA-capable toy sparse-routing experiment with straight-through Gumbel top-k supports. The inequality held for all tested q values, and q-ary regularization reduced severe overlap versus the linear control (q=2: 3.5% vs 5.1%; q=4: 2.75% vs 5.1%; q=8: 4.38% vs 5.1%), with the expected exponential sensitivity in the measured moment. It was not an overall win because the task objective degraded substantially relative to the unregularized baseline and no perplexity/generalization benefit was established.", "metrics": { "baseline": "No regularizer: task=0.00049, mean overlap=3.976, severe overlap >=3=100%; q=2 moment-1=14.81, q=4=250.44, q=8=4009.88.", "idea": "q-ary: q=2 task=0.1663, mean overlap=1.121, severe=3.50%, moment-1=1.54; q=4 task=0.3613, mean overlap=1.114, severe=2.75%, moment-1=7.31; q=8 task=0.4370, mean overlap=1.191, severe=4.38%, moment-1=43.93. Linear control task=0.1306, mean overlap=1.183, severe=5.13%." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 qary_overlap_experiment.py", "files": [ "qary_overlap_experiment.py", "results.json", "article.md", "idea_context.json" ], "limitations": "This is a tiny synthetic routing optimization, not a Transformer, WikiText-2, MoE, or GNN benchmark. Only one fixed seed and one regularization weight were tested; perplexity, masking robustness, gradient-noise scale, FLOPs, and multi-seed statistical significance were not measured." }