Эксперимент: Certified ambiguity gating for LLM supervision (#1412)

{ "worked": false, "confidence": 8, "verdict": "Built a self-contained certified ambiguity audit and toy student-training comparison in ambiguity_experiment.py. The finite-population construction reproduced the claimed minimax floor exactly: D/2=0.1885 for D=0.377, and the audit produced Dhat=0.3783 with a 0.1747 lower confidence bound. Ambiguity gating achieved 0.8952 accuracy and 0.0587 ECE, identical to matched confidence filtering and below hard-all (0.8994); soft labels reached 0.9066 but are not evidence that hard gating wins.", "metrics": { "baseline": "hard_all: accuracy 0.8994, ECE 0.1199; confidence_matched: accuracy 0.8952, ECE 0.0587", "idea": "ambiguity_gate: accuracy 0.8952, ECE 0.0587, retention 62.17%; soft_all: accuracy 0.9066, ECE 0.1326" }, "how_to_run": "/home/maxwelhelp/main/bin/python3 ambiguity_experiment.py", "files": [ "ambiguity_experiment.py", "results.json" ], "limitations": "This is a synthetic binary classification experiment, not a real frozen LLM, production verbalizer, or natural-language dataset. Only one fixed seed and one toy data geometry were tested; the strengthened exhaustive-grid check was not incorporated after the tool protocol failure, although the direct fair-random construction already exactly matched D/2." }