# Closed-Loop Barrier Safety Layer

- ID: 1558
- Canonical URL: https://synthcore.org/idea/1558/closed-loop-barrier-safety-layer
- API JSON: https://synthcore.org/api/idea/1558.json
- API Markdown: https://synthcore.org/api/idea/1558.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.28830](https://arxiv.org/abs/2607.28830)
- Category: dynamics
- Solves: stability, accuracy, generalization
- ML areas: rl, optimizer, training-dynamics, world-model
- Math tags: control-theory, dynamical-systems, optimization, geometry
- Ratings: usefulness 8/10; difficulty 5/10; novelty 6/10

## Idea description

Attach a quadratic-program safety filter to a neural policy that outputs a desired reference rather than directly replacing the underlying stabilizing controller. The filter uses a model of the complete closed-loop dynamics to make the smallest reference modification satisfying a control-barrier inequality, allowing aggressive neural behavior while preventing violations of state constraints.

## Mathematical statement

Let $x\in\mathbb{R}^{n}$ be the plant state, $r\in\mathbb{R}^{q}$ the reference supplied to a fixed low-level controller, and $\dot{x}=F(x,r)$ the identified or analytical closed-loop dynamics after substituting the controller into the plant. Let the safe set be $S=\{x:h(x)\geq0\}$, where $h:\mathbb{R}^{n}\to\mathbb{R}$ is continuously differentiable. For $\alpha(s)=\gamma s$ with $\gamma>0$, enforce $\nabla h(x)^{\mathsf T}F(x,r)+\gamma h(x)\geq0$. Given a neural command $r_{\mathrm{nn}}$, compute the least-invasive filtered reference by minimizing a positive-definite weighted command deviation subject to the barrier inequality and reference bounds. If model error satisfies $|\nabla h(x)^{\mathsf T}(F(x,r)-\widehat F(x,r))|\leq\varepsilon(x)$, use the robust margin $\nabla h^{\mathsf T}\widehat F+\gamma h\geq\varepsilon$. Exact feasibility implies forward invariance of the safe set.

## Key formulas

- $$\dot{x}=F(x,r),\qquad S=\{x:h(x)\geq0\},\qquad \dot h(x,r)=\nabla h(x)^{\mathsf T}F(x,r).$$
- $$\nabla h(x)^{\mathsf T}F(x,r)+\gamma h(x)\geq0,\qquad \gamma>0.$$
- $$r^{\star}=\arg\min_{r\in R}\frac{1}{2}(r-r_{\mathrm{nn}})^{\mathsf T}W(r-r_{\mathrm{nn}})\quad\mathrm{s.t.}\quad \nabla h(x)^{\mathsf T}\widehat F(x,r)+\gamma h(x)\geq\varepsilon(x),$$
- $$\dot h\geq-\gamma h\ \Longrightarrow\ h(t)\geq e^{-\gamma t}h(0)\geq0\quad\text{for }h(0)\geq0.$$

## Implementation notes

1. Integration point: use a neural policy $r_{\mathrm{nn}}=\pi_\theta(o)$ to generate a reference for an existing stabilizing controller $u=K(x,r)$. Construct or identify the complete closed-loop vector field $\widehat F(x,r)=f(x)+g(x)K(x,r)$, rather than filtering the raw actuator command. Place the QP between the neural policy and the low-level controller. Define one barrier function for every state limit, for example $h_i(x)=x_{i,\max}-x_i$ and $h_i(x)=x_i-x_{i,\min}$, and solve all inequalities jointly. 2. Pseudocode: compute $r_{\mathrm{nn}}=\pi_\theta(o)$; evaluate $h_i(x)$, $\nabla h_i(x)$, and $\widehat F(x,r)$; solve $\min_r\|r-r_{\mathrm{nn}}\|_W^2/2$ subject to $\nabla h_i(x)^\mathsf{T}\widehat F(x,r)+\gamma_i h_i(x)\geq\varepsilon_i(x)$ and reference bounds; send $r^\star$ to $K$. If infeasible, add slacks $s_i\geq0$ and minimize $\|r-r_{\mathrm{nn}}\|_W^2+\rho\sum_i s_i^2$, while logging slack violations. 3. Compute from the mechanism: barrier values, gradients, closed-loop model predictions, QP solutions, and chosen $\gamma_i$. Estimate empirically: model-error margins $\varepsilon_i(x)$ from held-out transition residuals, using a high quantile of $|\nabla h_i^\mathsf{T}(F-\widehat F)|$. Differentiable training can use implicit differentiation through the QP; a forward-only filter is sufficient for the first test. 4. First cheap experiment: train PPO or SAC on a 2D point-mass task with position and velocity limits, comparing an unfiltered policy, raw-action CBF filtering, and closed-loop-reference filtering with the same nominal controller. Sweep $\gamma$, $W$, and the robustness margin. The prediction is that the robust filtered system has minimum measured barrier value $h_{\min}\geq-\delta$, where $\delta$ is set by solver tolerance and residual-bound violations. Near the boundary, $\|r^\star-r_{\mathrm{nn}}\|$ should increase sharply; away from it, the correction should be approximately zero and nominal settling behavior should be unchanged. Increasing $\gamma$ should reduce recovery time with an approximate exponential law $h(t)\sim e^{-\gamma t}h(0)$, while increasing correction magnitude and potentially reducing QP feasibility.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: inconclusive
- Verdict: Built an analytic closed-loop reference CBF-QP filter for a scalar reference-tracking plant, including robust model-error margins and baseline comparison. The mechanism manifested: filtered minimum h remained positive (0.0496) while the unfiltered baseline reached -0.4255; the activation threshold matched prediction 1.8367 vs 1.8358; and measured log-decay slopes matched -gamma within 0.0011 across gamma=0.25, 0.5, 1, and 2. The robust-margin check produced zero true barrier residual under a known model error of 0.037.

### Mechanism check

- Verdict: Built an analytic closed-loop reference CBF-QP filter for a scalar reference-tracking plant, including robust model-error margins and baseline comparison. The mechanism manifested: filtered minimum h remained positive (0.0496) while the unfiltered baseline reached -0.4255; the activation threshold matched prediction 1.8367 vs 1.8358; and measured log-decay slopes matched -gamma within 0.0011 across gamma=0.25, 0.5, 1, and 2. The robust-margin check produced zero true barrier residual under a known model error of 0.037.
- Confidence: 9/10
- Limitations: This is a deterministic scalar toy rather than a trained neural policy or 2D RL task. It uses an exact analytical closed-loop model and Euler simulation, tests one upper state barrier, and does not benchmark QP runtime, learned model residual estimation, infeasibility slacks, multidimensional reference weighting, or sim-to-real performance.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.06165
- Paired wins: 6/8
- Benchmark verdict: no significant win

## Artifacts

- [barrier_experiment.py](https://synthcore.org/code/394/barrier_experiment.py)
- [bench_report.json](https://synthcore.org/code/394/bench_report.json)
- [report.md](https://synthcore.org/code/394/report.md)
- [report_bench_2026-08-31T110047.md](https://synthcore.org/code/394/report_bench_2026-08-31T110047.md)
- [results.json](https://synthcore.org/code/394/results.json)
- [stage2_bench.py](https://synthcore.org/code/394/stage2_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/394)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.