Koopman-generator HJB critic

1. Integration point: use this as a critic and differentiable model-learning module for a small continuous-control agent, or as a standalone neural optimal controller. Learn a dictionary encoder $\psi_\omega(x)$, drift model $\hat f_\eta(x)$, control model $\hat G_\eta(x)$, and value network $V_\theta(x)$. Evaluate the HJB residual at sampled states using automatic differentiation through $V_\theta$. 2. Pseudocode: collect tuples $(x_t,u_t,x_{t+\Delta})$; calculate $y_t=(x_{t+\Delta}-x_t)/\Delta$; regress $y_t\approx\hat f_\eta(x_t)+\hat G_\eta(x_t)u_t$; optionally calculate lifted differences $z_t=(\psi_\omega(x_{t+\Delta})-\psi_\omega(x_t))/\Delta$ and fit $z_t\approx A\psi_\omega(x_t)+B(x_t)u_t$; sample a minibatch of states; compute $g=\nabla_xV_\theta(x)$ and $u=-R^{-1}\hat G_\eta(x)^{\mathsf T}g$; compute $r=q(x)-\rho V_\theta(x)+g^{\mathsf T}(\hat f_\eta+\hat G_\eta u)+\tfrac12u^{\mathsf T}Ru$; update all parameters with $L=\mathbb{E}[r^2]+\lambda_d\mathbb{E}\|x_{t+\Delta}-x_t-\Delta(\hat f_\eta+\hat G_\eta u_t)\|^2+\lambda_b\mathbb{E}\|u\|^2$. 3. The mathematical component is the Koopman operator/generator identity and the HJB minimizing-control formula. Estimate finite-difference derivatives, dictionary dimension, model errors, and regularization weights empirically; freeze the dynamics model initially to prevent value fitting from corrupting identification, then optionally alternate model and value updates. 4. First experiment: compare against a standard MLP actor-critic and neural-ODE actor-critic on a two-dimensional Van der Pol, pendulum, or linear-quadratic-control benchmark using identical trajectory and parameter budgets. Track HJB residual, held-out one-step prediction error, episodic return, control energy, and the fraction of random initial states that remain bounded. Success means lower residual and fewer unstable trajectories at equal data; a useful target is at least 10% higher return or 2x lower one-step error without increasing parameter count by more than 25%.