Эксперимент: Koopman-generator HJB critic (#861)

{ "worked": true, "confidence": 9, "verdict": "Built a compact Koopman-generator/HJB critic experiment for a discounted continuous-time LQR system, including finite-difference generator estimation, control-affine regression, EDMD-style observable lifting, HJB residual training, and a TD baseline. All three required mechanism predictions manifested: generator error scaled as dt^0.995 versus predicted dt^1, identification error scaled as N^-0.535 versus predicted N^-0.5, and HJB residual was approximately zero with exact dynamics and scaled as |epsilon|^1.029 versus predicted first order. In the small neural comparison, HJB value error was 0.203 versus 3.093 for TD, but this is only a toy controlled benchmark and is not evidence of general RL superiority.", "metrics": { "baseline": "TD critic RMS value error 3.0930 after 220 updates", "idea": "HJB critic RMS value error 0.2034 after 220 updates; generator slope 0.9948, identification slope -0.5348, HJB dynamics-error slope 1.0285, exact-dynamics HJB residual 9.1e-16, lifted-generator holdout derivative RMS error 0.5134" }, "how_to_run": "OMP_NUM_THREADS=2 MKL_NUM_THREADS=2 /home/maxwelhelp/main/bin/python3 experiment.py", "files": [ "experiment.py", "results.json" ], "limitations": "Only a two-dimensional linear LQR toy system was tested; no nonlinear Van der Pol, pendulum, online data collection, learned dictionary encoder, long-horizon stability, wall-clock/FLOP matching, or statistically independent benchmark suite was evaluated. The EDMD lift was demonstrated but not used to train the value network; the neural comparison used exact known dynamics rather than a jointly learned world model." }