Эксперимент: Constant-sum ordinal preference loss (#1179)

{ "worked": false, "confidence": 9, "verdict": "Implemented the supplied adjacent-categories loss with constant-sum point parameterization, plus a paper-faithful category-logit AC variant and multiclass cross-entropy control. The recursion is numerically exact, but the supplied formula's equal-point-total likelihood gap changes from 0.25 to 1.45 as skill difference changes, so constant-sum points are not sufficient under that formulation. The paper-faithful AC model gives the expected skill-independent gap and only a negligible learning improvement over CE, so the supplied idea does not show the claimed effect.", "metrics": { "baseline": "Multiclass CE: mean held-out NLL 0.84228, accuracy 0.6332, pair-ranking Kendall tau 0.824 mean, with high seed instability.", "idea": "Supplied adjacent recursion: mean held-out NLL 0.85051, accuracy 0.6283, pair-ranking Kendall tau 0.8039. Adjacent-log-odds error was 4.4e-16, but the equal-score likelihood gap changed by 1.20 across skill differences. Paper-faithful constant-sum AC: mean NLL 0.84215, accuracy 0.6321, tau 0.8238; equal-score gap change was 0.0." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 experiment.py", "files": [ "experiment.py", "results.json" ], "limitations": "Only a small synthetic paired-comparison experiment was run. MovieLens, real repeated schedules, calibration, explicit score aggregation, FLOPs, and larger-scale generalization were not tested. The supplied adjacent formula and the paper's category-logit AC parameterization are mathematically different, so the paper-faithful results should not be attributed directly to the supplied formula." }