Executed-Action and Intervention-Aware Replay

1. Integration point: modify any off-policy actor-critic replay buffer. During collection, retain s, u_nom, u_exec, r, s_prime, barrier margin h, uncertainty residual e, and intervention norm I. During updates, feed u_exec to the critic; the actor can still be optimized through a differentiable safety layer or with a detached filter initially. 2. Pseudocode: insert the transition; compute q_i from the current TD error and stored safety statistics; sample with p_i proportional to (q_i plus epsilon_0)^alpha; apply importance weights; update critic and actor; recompute sampled TD errors and priorities. Start with alpha = 0.6 and beta = 0.4, then compare against uniform replay and ordinary TD-error prioritization. 3. Computed versus estimated: executed actions, intervention norms, and residuals are directly logged; TD errors are estimated by the current critic and priorities are stale between updates. Normalize each priority component by a running median or percentile so one term does not dominate. 4. First cheap experiment: train TD3 or SAC on a two-dimensional navigation task where a CBF modifies 5 to 30 percent of actions. Compare nominal-action critic plus uniform replay, executed-action critic plus uniform replay, and the full priority rule under equal environment steps. The prediction is that critic error on intervention transitions decreases faster and actor instability occurs at a larger intervention fraction. Sweep intervention probability and measure the crossover: executed-action training should outperform nominal-action training once more than about 10 percent of stored actions are altered. Also test whether the priority histogram shifts toward high-residual and high-intervention samples and whether success improves without increasing empirical contact rate.