# HOCBF Safety Shield for Neural Policies

- ID: 2448
- Canonical URL: https://synthcore.org/idea/2448/hocbf-safety-shield-for-neural-policies
- API JSON: https://synthcore.org/api/idea/2448.json
- API Markdown: https://synthcore.org/api/idea/2448.md
- Verification status: failed_benchmark
- Source: [arXiv:2608.17259](https://arxiv.org/abs/2608.17259)
- Category: dynamics
- Solves: stability, accuracy, generalization
- ML areas: rl, world-model, training-dynamics, regularization
- Math tags: control-theory, dynamical-systems, optimization, geometry
- Ratings: usefulness 7/10; difficulty 5/10; novelty 6/10

## Idea description

Use a neural policy only to generate a nominal action, then project that action onto the set satisfying a high-order control-barrier inequality derived from a smooth obstacle-distance function. This preserves the policy's behavior away from obstacles while enforcing a forward-invariant safety region near obstacles, and it can be used either as an inference-time shield or as a differentiable training layer.

## Mathematical statement

Let the system be control-affine, $\dot{x}=f(x)+G(x)u$, with state $x$, drift $f$, and input matrix $G$. Let $h(x)>0$ define the safe set, for example $h(p)=d(p,\mathcal{O})-d_{\mathrm{safe}}$, where $p$ is position, $\mathcal{O}$ is the obstacle set, and $d_{\mathrm{safe}}$ is the safety margin. If $h$ has relative degree two, define $\psi_0=h$, $\psi_1=\dot h+\alpha_1(h)$, and require $\psi_2=\dot\psi_1+\alpha_2(\psi_1)\geq0$, where $\alpha_1$ and $\alpha_2$ are strictly increasing class-$\mathcal{K}$ functions, commonly $\alpha_i(z)=k_i z$ with $k_i>0$. Since $\psi_2=L_f^2h+L_G L_fh\,u+\alpha_1'(h)L_fh+\alpha_2(\psi_1)$, the safety constraint is affine in $u$. The neural action $u_\pi=\pi_\theta(o)$ is replaced by the closest feasible action $u^*=\arg\min_u\frac12\|u-u_\pi\|_R^2$ subject to $\psi_2(x,u)\geq0$ and actuator bounds. Forward invariance follows when the model is correct, the initial state satisfies $h\geq0$ and $\psi_1\geq0$, and the constraint remains feasible.

## Key formulas

- $$\psi_0(x)=h(x),\qquad \psi_1(x)=\dot h(x)+\alpha_1(h(x)),\qquad \psi_2(x,u)=\dot\psi_1(x,u)+\alpha_2(\psi_1(x)).$$
- $$\psi_2=L_f^2h(x)+L_G L_fh(x)u+\alpha_1'(h(x))L_fh(x)+\alpha_2(\psi_1(x))\geq0.$$
- $$u^*=\arg\min_{u}\frac12(u-u_\pi)^TR(u-u_\pi)\quad\mathrm{s.t.}\quad \psi_2(x,u)\geq0,\quad u_{\min}\leq u\leq u_{\max}.$$
- $$\dot h\geq-k_1h\ \text{and}\ \psi_2\geq0\ \Longrightarrow\ h(t)\geq0\ \text{for all }t\geq0\ \text{when }h(0)\geq0,\ \psi_1(0)\geq0.$$

## Implementation notes

1. Integration point: attach the shield after a policy network and before the environment or actuator. For a first implementation, use a 2D point-mass or quadrotor position controller with state $x=(p,v)$ and known dynamics $\dot p=v$, $\dot v=a$, where $a$ is the policy output; later replace $a$ with thrust and attitude commands using the quadrotor model. Define each obstacle's smooth barrier as $h_j(p)=\sqrt{\|p-c_j\|^2+\epsilon^2}-r_j-d_{\mathrm{safe}}$, where $c_j$ and $r_j$ are obstacle center and radius and $\epsilon>0$ smooths the distance. 2. Pseudocode: compute $u_\pi=\pi_\theta(o)$; evaluate $h_j$, $\dot h_j$, $L_f^2h_j$, and $L_G L_fh_j$ from the differentiable dynamics model; solve a small quadratic program minimizing $(u-u_\pi)^TR(u-u_\pi)$ subject to $L_f^2h_j+L_G L_fh_j u+k_1\dot h_j+k_2(\dot h_j+k_1h_j)\geq0$ for every obstacle and actuator bounds; execute $u^*$. During training, backpropagate through the analytic QP layer or use the shielded action while logging the projection distance. 3. Computed quantities are barrier derivatives and constraints from the control model; estimate model error empirically from one-step prediction residuals and optionally tighten each constraint by $\delta=\beta\hat\sigma(x)$, where $\hat\sigma$ is a residual standard deviation. 4. First cheap experiment: train PPO or SAC on a 2D obstacle-navigation task with an MLP policy, comparing no shield, a penalty-only baseline, and the HOCBF shield. Use $k_1=k_2=2$, $R=I$, and sweep initial distances and policy action scales. The falsifiable signature is a sharp feasibility and violation transition: as nominal action magnitude increases, violations should remain near zero while the QP is feasible, then rise abruptly when the fraction of infeasible states becomes nonzero. For correctly modeled dynamics, the measured minimum barrier should satisfy $\min_t h_j(t)\geq0$ up to discretization error; with timestep $\Delta t$, halve $\Delta t$ and verify that minimum-barrier violation decreases approximately quadratically for the second-order position integrator. The activation rate should grow primarily when $h_j$ or $\psi_{1,j}$ approaches zero, while navigation reward away from obstacles should remain within 5% of the unshielded policy.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: no_effect
- Verdict: Built an analytic 2-D double-integrator HOCBF safety shield with actuator-bounded Euclidean projection and reproducible numerical experiments. The derivative identities were confirmed with maximum absolute errors of 2.7e-6 for h-dot and 1.4e-5 for psi1-dot; required normal acceleration scaled exactly linearly with k2, with observed slope -0.170955 versus predicted -psi1=-0.170955. Shield activation began at the predicted boundary transition, and shielded rollouts maintained positive minimum h while unshielded rollouts collided. This demonstrates the mechanism in the toy setting, but not improved learned-policy reward.

### Mechanism check

- Verdict: Built an analytic 2-D double-integrator HOCBF safety shield with actuator-bounded Euclidean projection and reproducible numerical experiments. The derivative identities were confirmed with maximum absolute errors of 2.7e-6 for h-dot and 1.4e-5 for psi1-dot; required normal acceleration scaled exactly linearly with k2, with observed slope -0.170955 versus predicted -psi1=-0.170955. Shield activation began at the predicted boundary transition, and shielded rollouts maintained positive minimum h while unshielded rollouts collided. This demonstrates the mechanism in the toy setting, but not improved learned-policy reward.
- Confidence: 8/10
- Limitations: Only a known-model 2-D point-mass system and one fixed nominal policy were tested. No PPO/SAC training, differentiable QP backpropagation, model uncertainty, multiple obstacles, disturbances, or repeated statistical trials were evaluated; discrete-time safety results are not a formal continuous-time guarantee.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.3172
- Paired wins: 5/8
- Benchmark verdict: no measurable effect

## Artifacts

- [bench_hocbf.py](https://synthcore.org/code/949/bench_hocbf.py)
- [bench_report.json](https://synthcore.org/code/949/bench_report.json)
- [hocbf_experiment.py](https://synthcore.org/code/949/hocbf_experiment.py)
- [report.md](https://synthcore.org/code/949/report.md)
- [report_bench_2026-09-01T164201.md](https://synthcore.org/code/949/report_bench_2026-09-01T164201.md)
- [results.json](https://synthcore.org/code/949/results.json)
- [Download all files as ZIP](https://synthcore.org/download/949)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.