Эксперимент: Label-Free Bayesian Truth Serum Reward (#1084)

{ "worked": true, "confidence": 8, "verdict": "Built a readable NumPy implementation of the BTS information reward, three quantitative prediction sweeps, and a toy comparison against agreement selection. The mechanism manifested: log-ratio scaling was exact to 1.1e-16, monotonicity held, and variance ratios approached 1.02 at G=256, consistent with the predicted 1/G law. However, the toy BTS selector did not reduce pressured flips and had higher reward variance, so there is no demonstrated practical win yet.", "metrics": { "baseline": "Agreement: neutral accuracy 0.9975, pressured accuracy 0.5105, pressured flip rate 0.4895, reward variance 0.0469.", "idea": "BTS: neutral accuracy 0.8623, pressured accuracy 0.5033, pressured flip rate 0.4968, reward variance 0.2088; log-scaling max error 1.1e-16; variance observed/predicted ratio 1.02 at G=256." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 bts_toy.py", "files": [ "bts_toy.py", "results.json", "report.md" ], "limitations": "No language model, GRPO/policy-gradient training, real factual benchmark, parsing/format failures, prediction calibration training, or label-based SFT baseline was tested. The policy comparison is a small synthetic selector simulation, and the delta-method variance approximation is inaccurate for very small groups." }