HOCBF Safety Shield for Neural Policies

1. Integration point: attach the shield after a policy network and before the environment or actuator. For a first implementation, use a 2D point-mass or quadrotor position controller with state $x=(p,v)$ and known dynamics $\dot p=v$, $\dot v=a$, where $a$ is the policy output; later replace $a$ with thrust and attitude commands using the quadrotor model. Define each obstacle's smooth barrier as $h_j(p)=\sqrt{\|p-c_j\|^2+\epsilon^2}-r_j-d_{\mathrm{safe}}$, where $c_j$ and $r_j$ are obstacle center and radius and $\epsilon>0$ smooths the distance. 2. Pseudocode: compute $u_\pi=\pi_\theta(o)$; evaluate $h_j$, $\dot h_j$, $L_f^2h_j$, and $L_G L_fh_j$ from the differentiable dynamics model; solve a small quadratic program minimizing $(u-u_\pi)^TR(u-u_\pi)$ subject to $L_f^2h_j+L_G L_fh_j u+k_1\dot h_j+k_2(\dot h_j+k_1h_j)\geq0$ for every obstacle and actuator bounds; execute $u^*$. During training, backpropagate through the analytic QP layer or use the shielded action while logging the projection distance. 3. Computed quantities are barrier derivatives and constraints from the control model; estimate model error empirically from one-step prediction residuals and optionally tighten each constraint by $\delta=\beta\hat\sigma(x)$, where $\hat\sigma$ is a residual standard deviation. 4. First cheap experiment: train PPO or SAC on a 2D obstacle-navigation task with an MLP policy, comparing no shield, a penalty-only baseline, and the HOCBF shield. Use $k_1=k_2=2$, $R=I$, and sweep initial distances and policy action scales. The falsifiable signature is a sharp feasibility and violation transition: as nominal action magnitude increases, violations should remain near zero while the QP is feasible, then rise abruptly when the fraction of infeasible states becomes nonzero. For correctly modeled dynamics, the measured minimum barrier should satisfy $\min_t h_j(t)\geq0$ up to discretization error; with timestep $\Delta t$, halve $\Delta t$ and verify that minimum-barrier violation decreases approximately quadratically for the second-order position integrator. The activation rate should grow primarily when $h_j$ or $\psi_{1,j}$ approaches zero, while navigation reward away from obstacles should remain within 5% of the unshielded policy.