# Adaptive CBF Safety Layer for Neural Policies

- ID: 1335
- Canonical URL: https://synthcore.org/idea/1335/adaptive-cbf-safety-layer-for-neural-policies
- API JSON: https://synthcore.org/api/idea/1335.json
- API Markdown: https://synthcore.org/api/idea/1335.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.23839](https://arxiv.org/abs/2607.23839)
- Category: dynamics
- Solves: stability, accuracy, generalization
- ML areas: rl, mlp, training-dynamics
- Math tags: control-theory, dynamical-systems, optimization
- Ratings: usefulness 8/10; difficulty 5/10; novelty 6/10

## Idea description

Wrap a neural policy with a safety filter that minimally modifies its action so that a control-barrier inequality remains satisfied under bounded model mismatch and actuator saturation. Estimate mismatch between a learned plant or reference model and observed transitions online, then enlarge a conservative error margin and shrink the admissible safe set before solving the filter. The neural policy is unchanged when its action is safe, but receives a principled correction near state or action constraints.

## Mathematical statement

Consider a control-affine system dot{x} = f(x) + g(x)u + d(x,t), where x is the state, u is the action, and d is unknown model error. Let the safe set be C = {x : h(x) >= 0}, with differentiable barrier h. For a relative-degree-one constraint, forward invariance is certified by dot{h}(x,u) + alpha h(x) >= 0, where alpha > 0 and dot{h} = grad h(x)^T(f(x) + g(x)u). If the projected disturbance satisfies |grad h(x)^T d(x,t)| <= epsilon_h(x), use the robust condition dot{h}_model + alpha h - epsilon_h >= 0. Implement the paper's error-based adaptation with e_t = x_{t+1} - xhat_{t+1}, an exponentially smoothed mismatch ebar_t = rho ebar_{t-1} + (1-rho)||e_t||, and adaptive barrier h_a(x,t) = h(x) - kappa ebar_t. The action also obeys componentwise saturation u_min <= u <= u_max.

## Key formulas

- $$\dot{x}=f(x)+g(x)u+d(x,t),\qquad \mathcal{C}=\{x:h(x)\geq0\}.$$
- $$\dot{h}_{\mathrm{model}}(x,u)+\alpha h(x)-\varepsilon_h(x)\geq0,\qquad \dot{h}_{\mathrm{model}}=\nabla h(x)^{T}\bigl(f_{\theta}(x)+g_{\theta}(x)u\bigr).$$
- $$h_a(x,t)=h(x)-\kappa\bar e_t,\qquad \bar e_t=\rho\bar e_{t-1}+(1-\rho)\left\|x_{t+1}-\hat{x}_{t+1}\right\|_2.$$
- $$u_t=\arg\min_u\;\frac{1}{2}\left\|u-\pi_{\phi}(x_t)\right\|_R^2+\lambda_s s^2\quad\text{subject to}\quad \dot{h}_{a,\mathrm{model}}(x_t,u)+\alpha h_a(x_t,t)\geq-s,\;u_{\min}\leq u\leq u_{\max},\;s\geq0.$$

## Implementation notes

1. Integration point: place the safety filter after the neural policy head and before the environment or actuator. Begin with a discrete-time RL experiment using a two-layer MLP policy. Define known constraints such as h_j(x) = x_{j,max} - x_j and h_j(x) = x_j - x_{j,min}. Use either a known simulator model or a learned one-step model xhat_{t+1} = F_theta(x_t,u_t). 2. Pseudocode: compute u_nom = pi_phi(x_t); predict xhat_{t+1} = F_theta(x_t,u_nom); after the next observation arrives, calculate e_t = ||x_{t+1} - xhat_{t+1}||; update ebar = rho*ebar + (1-rho)*e_t; set h_a = h - kappa*ebar; linearize each barrier through the model prediction; solve a small quadratic program minimizing deviation from u_nom subject to the adaptive barrier inequalities, action bounds, and nonnegative slack s. During training, use the filtered action in the environment and optionally penalize s and the intervention norm. 3. Computed versus estimated: h and its gradient are designed from the safety specification; F_theta or f_theta and g_theta are learned or estimated; ebar is measured online; alpha, rho, kappa, and lambda_s are hyperparameters. If a calibrated error quantile is available, replace kappa*ebar by that quantile projected onto each barrier gradient. 4. First cheap experiment: train PPO with a two-layer MLP on constrained CartPole, Safety Gymnasium Point, or a pendulum with angle and torque limits. Compare PPO, PPO with action clipping, and PPO with this QP layer. Create model mismatch by training the safety model on 10%, 25%, 50%, and 100% of the dynamics data. The quantitative prediction is a safety-margin transition: empirical violations should remain near zero while kappa*ebar exceeds the true one-step barrier error, and violations should rise when the observed error quantile exceeds the adaptive margin. At fixed mismatch, increasing alpha should reduce barrier recovery time approximately exponentially at rate alpha, while increasing kappa should monotonically reduce violations and increase intervention magnitude. Falsify the mechanism if the adaptive filter does not improve the violation-versus-return Pareto frontier over static action clipping.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built a scalar adaptive CBF safety filter with bounded action saturation, exponentially smoothed model mismatch, adaptive barrier shrinkage, and a closed-form QP projection. The robust-residual check passed, exponential smoothing predictions matched observations exactly, and predicted adaptive-margin crossing times matched for kappa > 1. Increasing kappa monotonically increased intervention and reduced violations from 87.9 at kappa=0 to 0 at kappa>=1, outperforming static clipping, so the mechanism manifested in this toy setting.

### Mechanism check

- Verdict: Built a scalar adaptive CBF safety filter with bounded action saturation, exponentially smoothed model mismatch, adaptive barrier shrinkage, and a closed-form QP projection. The robust-residual check passed, exponential smoothing predictions matched observations exactly, and predicted adaptive-margin crossing times matched for kappa > 1. Increasing kappa monotonically increased intervention and reduced violations from 87.9 at kappa=0 to 0 at kappa>=1, outperforming static clipping, so the mechanism manifested in this toy setting.
- Confidence: 8/10
- Limitations: This is a scalar deterministic toy experiment, not PPO, CartPole, a learned neural dynamics model, or a full numerical QP. It does not test stochastic disturbances, multidimensional barrier intersections, model retraining, return/performance tradeoffs, or calibrated error quantiles; the persistent boundary-seeking nominal policy also makes intervention costs unusually large.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [adaptive_cbf_experiment.py](https://synthcore.org/code/356/adaptive_cbf_experiment.py)
- [bench_report.json](https://synthcore.org/code/356/bench_report.json)
- [report.md](https://synthcore.org/code/356/report.md)
- [report_bench_2026-08-31T091810.md](https://synthcore.org/code/356/report_bench_2026-08-31T091810.md)
- [results.json](https://synthcore.org/code/356/results.json)
- [stage2_adaptive_cbf_bench.py](https://synthcore.org/code/356/stage2_adaptive_cbf_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/356)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.