Эксперимент: Feasibility-Ranked Group Policy Gradient (#924)

{ "worked": true, "confidence": 8, "verdict": "Built and ran a self-contained feasibility-ranked group policy-gradient MVP with timestep-wise normalization, clipped ratios, and feasible/violating trajectory weights. The mechanism manifested: normalized variance matched the exact prediction across scales (0.9999996–0.999999999), and observed feasibility-margin values tracked analytical predictions closely across return separations and weight ratios. In the toy policy experiment, ranking increased final feasibility from 2.20e-9 to 3.21e-5 and reduced risky-action probability from 0.9815 to 0.8741, at the expected cost of lower unconstrained reward.", "metrics": { "baseline": "Standard normalized group-relative update: final feasibility 2.20e-9, risky probability 0.9815, expected reward 5.9629.", "idea": "Feasibility-ranked update (wF=3, wV=0.25): final feasibility 3.21e-5, risky probability 0.8741, expected reward 5.7481. Normalized variance prediction error was below 4e-7; margin predictions were generally within about 2% for nonzero separation." }, "how_to_run": "/home/maxwelhelp/main/bin/python3 experiment.py", "files": [ "experiment.py", "results.json", "rerun.json" ], "limitations": "The policy comparison is a tiny synthetic Bernoulli horizon rather than CartPole/Pendulum, and does not compare against a learned-critic PPO implementation. The feasibility gain is absolute but both methods remain poor under the deliberately conflicting reward/constraint setup; no larger-scale sample-efficiency or wall-clock study was performed." }