Actuator-Aware Envelope Scheduler

Place the scheduler between a neural policy $\pi_\theta(s_k)$ and the physical or simulated actuator. Produce raw action $a_k=\pi_\theta(s_k)$, map it to $u_k=\bar u\tanh(a_k)$, and apply a rate limiter $u_k\leftarrow u_{k-1}+\operatorname{clip}(u_k-u_{k-1},-\bar r\Delta,\bar r\Delta)$. Maintain $r_k$ per trajectory for tracking error, action error, or model-prediction error. Compute the violation signal $v_k$, update $r_k$ with the projected adaptive formula, and add $\lambda_{env}\operatorname{softplus}(|e_k|/r_k-1+\epsilon)$ to the policy loss. Initially detach $r_k$ from gradients so the policy cannot exploit the scheduler; later compare with differentiating through it. The adaptive envelope principle and actuator-feasibility separation come from the paper, while the scalar update is an implementable discrete realization; $l_{tight}$, $l_{relax}$, bounds, and loss weight are empirically selected. First test a small MLP policy on bounded pendulum swing-up or 1D point-mass tracking with $\bar u=1$ and $\bar r=0.2$. Compare ordinary clipping, fixed-envelope regularization, and adaptive envelopes. Measure tracking error, clipping fraction, envelope violations, and recovery time after saturation. The predicted transition is that persistent clipping with insufficient $l_{relax}$ causes repeated envelope violations, while increasing $l_{relax}$ restores bounded normalized error and lowers sustained clipping.