Uncertainty-Inflated CBF Safety Layer
Implementation & benchmark of arXiv:2608.04732 — Toward Integrating Adaptive Experience Replay and Online Uncertainty Estimation in Safe Actor-Critic Optimal Control
Source paper: Toward Integrating Adaptive Experience Replay and Online Uncertainty Estimation in Safe Actor-Critic Optimal Control arXiv:2608.04732 ⓘ · analyzed Aug 31, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Attach an online uncertainty estimator to the perception or dynamics model and inflate every obstacle constraint by a confidence radius before applying the control-barrier-function filter. The actor still proposes the nominal action, but the executed action is the closest admissible action satisfying the uncertainty-adjusted barrier inequality, producing a tunable safety-versus-intervention mechanism.
Formulas
Mathematical statement
Let the true safety function be h(x) >= 0, its estimate be h_hat(x), and its estimation error be e_h(x) = h(x) - h_hat(x). If the estimator supplies a high-probability bound |e_h(x)| <= epsilon(x), then enforcing h_hat(x) >= epsilon(x) implies h(x) >= 0. For relative-degree-one dynamics x_dot = f(x) + g(x)u + d, use the robust CBF constraint grad(h_hat)(x)^T [f(x) + g(x)u] + alpha [h_hat(x) - epsilon(x)] >= ||grad(h_hat)(x)|| delta_max, where alpha > 0 is a class-K gain and delta_max bounds the disturbance projection. The safety filter solves a quadratic program minimizing deviation from the actor action subject to this constraint and actuator bounds.
Implementation notes
1. Integration point: place the module between the actor and the environment. The actor outputs u_nom = pi_theta(o); an uncertainty head outputs either a standard deviation sigma(o) or an ensemble variance for obstacle distance and, optionally, dynamics residuals. Convert it to a margin epsilon(o) = k sigma(o), with k selected from a calibration quantile. 2. Pseudocode: estimate h_hat, grad(h_hat), and epsilon; form the robust CBF linear inequality; solve a small quadratic program for u_exec; step the environment with u_exec; store both u_nom and u_exec. Train the uncertainty head from held-out sensor truth or ensemble disagreement. 3. Computed versus estimated: f, g, grad(h_hat), and the quadratic program are computed online; epsilon, delta_max, and calibration coverage are estimated from validation residuals. If no analytical dynamics model exists, finite-difference or learned Jacobians can be used, but this weakens the certificate. 4. First cheap experiment: use SAC on a two-dimensional point-mass navigation task with Gaussian obstacle-position noise, comparing no filter, nominal CBF, and uncertainty-inflated CBF. Sweep k in {0, 1, 2, 3} and several noise levels. The prediction is a sharp reduction in contacts once k sigma exceeds the empirical 95th-percentile distance residual; for calibrated errors, contact probability should be below roughly 5 percent, while intervention rate and path cost increase monotonically with k. Measure whether the observed safe-set violation rate tracks the uncovered calibration fraction.
Verification
Stage 1 · Toy mechanism gate: Passed ✓
Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗
Stage 2 · Practical benchmark result: No measurable effect
Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works
Stage 1 — Mechanism check agent confidence 9/10
Built an exact 1-D uncertainty-inflated CBF safety layer with actuator-bounded QP projection, numerical math checks, and noisy obstacle rollouts. The safety implication had zero violations for all tested bounded-error margins; intervention began at the predicted epsilon threshold of 0.35, observed between 0.3 and 0.4, and had the predicted slope of 2.0. Increasing k reduced contacts monotonically while increasing intervention and filter cost, demonstrating the intended safety-versus-intervention mechanism.
- Agent confidence
- 9/10
- Baseline
- Nominal CBF contact rates were 42.7%, 48.0%, and 52.3% at sigma 0.02, 0.05, and 0.1; no-filter contact rate was 100%.
- Idea
- Robust CBF with k=2 gave contact rates of 3.0%, 2.7%, and 3.3% at sigma 0.02, 0.05, and 0.1, with intervention rates of 89.6%, 91.3%, and 93.5%. Predicted intervention slope was 2.0 and observed slope was 2.000.
Limitations: This is a toy 1-D relative-degree-one system rather than SAC or a 2-D navigation environment. The uncertainty is a fixed Gaussian obstacle-position error, not a learned online estimator or calibrated neural uncertainty head; multi-dimensional QPs, learned dynamics, sensor simulation, and long-horizon RL training were not tested.
How to run: python3 uncertainty_cbf_experiment.py > results.json
Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.04732)
No significant difference vs. the tuned baseline (−-0.0%, p=1, wins 0 / 8 paired seeds).
- Benchmark
- Dynamics forecasting
- Model
- rnn_small
- Paired seeds
- 8
- Baseline mean
- 0.0043
- Idea mean
- 0.0043
- Effect (Δ)
- 0 (−-0.0%; negative = idea better)
- Wins
- 0 / 8 paired seeds
- p-value
- 1 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- ±0.0%
- Mechanism
- Not confirmed ✗
- Practical effect
- No effect
- Baseline tuning
- swept over 3 configs
The official README file was absent despite the package and built-in track being importable; the available source API was used directly. The benchmark evaluates one-step supervised pendulum prediction rather than SAC or long-horizon closed-loop safety contacts. The current intervention is a simplified one-dimensional robust prediction/action transformation rather than a full online perception uncertainty estimator plus general multi-dimensional QP; only the registered dynamics task and the tested lr/k grids were evaluated.
How to run: python3 stage2_official.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Stage 2 — Benchmark latest Failed ✗
Agent confidence: 10/10
The idea was implemented and evaluated on the registered built-in dynamics track using rnn_small, with baseline tuning through sweep_baseline and final comparison through make_report. Baseline mean test MSE was 0.00428859; the best idea configuration was k=0, producing the identical metric, delta=0 and p=1.0. Positive inflation worsened performance (k=1 mean MSE 0.00978588; k=2 mean MSE 0.210504), and the trained-model mechanism signature was not confirmed because the selected setting had zero intervention and zero coverage.
- Baseline
- Registered track dynamics, rnn_small, best lr=0.004; sweep means: lr=0.001 -> 0.0220817, lr=0.002 -> 0.00584660, lr=0.004 -> 0.00429530; full 8-seed mean MSE=0.00428859.
- Idea
- Best k=0.0 at lr=0.004; full 8-seed mean MSE=0.00428859. Idea sweep: k=0 -> 0.00428859, k=1 -> 0.00978588, k=2 -> 0.210504. Paired delta=0, p=1.0, intervention=0, mechanism confirmed=false.
Benchmark result
No significant difference vs. the tuned baseline (−-0.0%, p=1, wins 0 / 8 paired seeds).
- Benchmark
- Dynamics forecasting
- Model
- rnn_small
- Paired seeds
- 8
- Baseline mean
- 0.0043
- Idea mean
- 0.0043
- Effect (Δ)
- 0 (−-0.0%; negative = idea better)
- Wins
- 0 / 8 paired seeds
- p-value
- 1 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- ±0.0%
- Mechanism
- Not confirmed ✗
- Practical effect
- No effect
- Baseline tuning
- swept over 3 configs
The official README file was absent despite the package and built-in track being importable; the available source API was used directly. The benchmark evaluates one-step supervised pendulum prediction rather than SAC or long-horizon closed-loop safety contacts. The current intervention is a simplified one-dimensional robust prediction/action transformation rather than a full online perception uncertainty estimator plus general multi-dimensional QP; only the registered dynamics task and the tested lr/k grids were evaluated.
How to run: python3 stage2_official.py
Artifacts
- bench_report.json 4.7 KB View Raw
- report.md 1.5 KB View
- report_bench_2026-08-31T213436.md 7.0 KB View
- results.json 4.5 KB View Raw
- stage2_bench.py 6.8 KB View Raw
- stage2_official.py 5.0 KB View Raw
- uncertainty_cbf_experiment.py 3.8 KB View Raw