Toward Integrating Adaptive Experience Replay and Online Uncertainty Estimation in Safe Actor-Critic Optimal Control
arXiv:2608.04732
2026
Dynamics
2 ideas extracted · analyzed Aug 31, 2026
What the math gives to ML
The paper offers a coupled safety-learning mechanism rather than a new standalone optimizer: online perception uncertainty enlarges the obstacle geometry used by a control-barrier-function filter, while filter interventions and estimation residuals control replay exposure. The critic is trained on the action actually executed after safety filtering, avoiding a mismatch between the actor's nominal action and the transition stored in the environment. The strongest transferable idea is an uncertainty-calibrated barrier margin with a measurable safety-versus-intervention tradeoff; the replay rule is a useful second mechanism, although its evidence is limited to a two-dimensional navigation benchmark.
Ideas from this paper
✗ Failed on benchmark
2026
Attach an online uncertainty estimator to the perception or dynamics model and inflate every obstacle constraint by a confidence radius before applying the control-barrier-function filter. The actor still proposes the nominal action, but the executed action is the closest admissible action satisfying the uncertainty-adjusted barrier inequality, producing a tunable safety-versus-intervention mechanism.
Useful7/10
Difficulty5/10
Novelty7/10
Unverified
2026
Train the critic on the action that the environment actually received after safety filtering, not only on the actor's nominal action. Prioritize transitions whose estimation residual, barrier proximity, or filter intervention is large, so replay concentrates on the distribution shift introduced by the safety controller instead of repeatedly sampling benign nominal behavior.
Useful6/10
Difficulty4/10
Novelty6/10