# Robust HOCBF Safety Shield for Neural Policies - ID: 2245 - Canonical URL: https://synthcore.org/idea/2245/robust-hocbf-safety-shield-for-neural-policies - API JSON: https://synthcore.org/api/idea/2245.json - API Markdown: https://synthcore.org/api/idea/2245.md - Verification status: mech_ok_no_baseline - Source: [arXiv:2608.13507](https://arxiv.org/abs/2608.13507) - Category: dynamics - Solves: stability, accuracy, generalization - ML areas: rl, world-model, training-dynamics - Math tags: control-theory, dynamical-systems, optimization, geometry, probability - Ratings: usefulness 8/10; difficulty 5/10; novelty 6/10 ## Idea description Wrap a neural policy with a small quadratic program that minimally modifies its acceleration or thrust command whenever predicted pairwise separation approaches a safety boundary. Use a learned residual model to estimate uncertainty and inflate the barrier constraint by a high-probability disturbance bound, giving a falsifiable safety-versus-control-authority tradeoff instead of relying on unconstrained policy behavior. ## Mathematical statement For agents with positions $p_i\in\mathbb{R}^d$, velocities $v_i$, and commanded accelerations $a_i$, define the pairwise safety function $h_{ij}(p)=\|p_i-p_j\|^2-d_{\mathrm{safe}}^2$, where $d_{\mathrm{safe}}>0$ is the required separation. Because $h_{ij}$ has relative degree two with respect to acceleration, use the exponentially stabilizing HOCBF recursion $\psi_{0,ij}=h_{ij}$ and $\psi_{1,ij}=\dot h_{ij}+\alpha_1h_{ij}$, where $\alpha_1>0$. The robust constraint is $\ddot h_{ij}+(\alpha_1+\alpha_2)\dot h_{ij}+\alpha_1\alpha_2h_{ij}\geq\delta_{ij}$, with $\alpha_2>0$ and uncertainty margin $\delta_{ij}\geq0$. If the acceleration residual satisfies $|e_{ij}^{\top}(w_i-w_j)|\leq\delta_{ij}$ with probability at least $1-\varepsilon$, then satisfying the robust constraint implies forward invariance of $h_{ij}\geq0$ with probability at least $1-\varepsilon$. Here $e_{ij}=p_i-p_j$, $w_i,w_j$ are model disturbances, and $\delta_{ij}$ is estimated from calibrated residual quantiles. The neural action is obtained by minimizing deviation from the nominal policy subject to all pairwise robust HOCBF inequalities and actuator bounds. ## Key formulas - $$h_{ij}(p)=\|p_i-p_j\|^2-d_{\mathrm{safe}}^2,\qquad \dot h_{ij}=2(p_i-p_j)^\top(v_i-v_j).$$ - $$\psi_{1,ij}=\dot h_{ij}+\alpha_1h_{ij},\qquad \ddot h_{ij}+(\alpha_1+\alpha_2)\dot h_{ij}+\alpha_1\alpha_2h_{ij}\geq\delta_{ij}.$$ - $$a^*=\arg\min_{a}\frac{1}{2}(a-a_{\mathrm{NN}})^\top W(a-a_{\mathrm{NN}})+\frac{\rho}{2}\|s\|^2\quad\mathrm{s.t.}\quad A_{\mathrm{CBF}}(p,v)a+s\geq b_{\mathrm{CBF}}(p,v)+\delta,\;a\in\mathcal{U},\;s\geq0.$$ - $$\delta_{ij}=Q_{1-\varepsilon}\left(\left|e_{ij}^{\top}(\widehat w_i-\widehat w_j)\right|\right).$$ ## Implementation notes 1. Exact integration point: let a neural policy produce nominal per-agent accelerations $a_{\mathrm{NN},i}=\pi_\theta(o_i,o_{\mathcal{N}(i)})$ in a multi-agent RL, imitation-learning, or learned-physics environment. Insert the QP after the policy and before the simulator or real actuator. For a quadrotor, convert the filtered desired force into thrust magnitude and attitude using the paper's thrust-attitude realization; for a simpler MVP, use a fully actuated point-mass simulator. 2. Pseudocode: compute $h_{ij}$ and $\dot h_{ij}$ for each neighbor pair; compute the nominal relative acceleration contribution in $\ddot h_{ij}$; form the affine inequality $A_{ij}a\geq b_{ij}+\delta_{ij}$; solve the strictly convex QP minimizing weighted action change; execute $a^*$ and log the correction norm, minimum separation, and solver slack. Add a large slack penalty only to diagnose infeasibility, not to claim formal safety. 3. Computed versus estimated: geometry, barrier derivatives, and QP matrices are computed exactly from observations and the known kinematic model. Estimate disturbance residuals on a held-out calibration set by comparing observed accelerations with the nominal learned model, and set $\delta_{ij}$ to an empirical $(1-\varepsilon)$ quantile, optionally conditioned on speed or state region. Validate calibration separately under distribution shift. 4. First cheap experiment: train PPO or behavior cloning on a 2-D eight-agent navigation task with circular obstacles and pairwise separation, comparing an unconstrained MLP policy, a nominal CBF-QP, and this robust HOCBF-QP. Sweep injected acceleration noise and the safety distance. The quantitative prediction is that violation probability remains below approximately $\varepsilon$ while the residual bound is calibrated, whereas the unconstrained policy violation rate rises sharply with noise. Also sweep the maximum acceleration $a_{\max}$: QP infeasibility or rapidly increasing action correction should appear when $a_{\max}$ is smaller than the required barrier correction, approximately when $\|A_{ij}a_{\max}\|