ISS-CLF/RCBF Neural Policy Shield

1. Integration point: use a neural policy u_theta(x) in an RL agent, an RNN/SSM controller, or a learned latent-dynamics controller. At every control step, evaluate the policy once and pass its action through a tiny QP layer. For an initial MVP, use a known two-dimensional plant such as a pendulum or point mass, choose V(x)=||x-x_goal||_2^2 and h(x) as the signed distance to obstacles or state limits, and use box constraints for u. 2. Pseudocode: compute f_0(x), g(x), and p(x); compute u_net=u_theta(x); compute gradients of V and h by autodiff; calculate r_V=w_bar ||p^T grad V||_2 and r_h=w_bar ||p^T grad h||_2; solve the displayed QP; execute u*. During training, backpropagate through a differentiable QP solver or stop gradients through the projection initially. 3. Computed quantities are the CLF/CBF gradients, nominal vector fields, actuator bounds, and analytic worst-case norm margins. Estimate w_bar from held-out model residuals using a high quantile, and log both slack variables and QP infeasibility. 4. First cheap experiment: train the same PPO or SAC MLP on a two-dimensional obstacle-avoidance task with and without the shield, using identical seeds and injected bounded disturbances. The quantitative prediction is that trajectories satisfying the CLF with negligible slack obey an empirical envelope V(t) approximately less than or equal to exp(-c_V t)V(0)+d_V/c_V; fit the decay rate and require it to agree with c_V within 20 percent. Increase w_bar until the robust margin crosses the available control authority; QP infeasibility and safety violations should exhibit a sharp transition near that predicted feasibility boundary. Compare action deviation, violation rate, and episode return.