Slow-MPC Fast-Policy Residual Control

1. Exact integration point: use this controller for continuous-control RL, model-based RL, or a learned-world-model environment. Partition the action into a slow nominal component and a fast residual component, or use the MPC output as an additional policy input. Normalize state and exogenous-input features. Run MPC only when $k\bmod M=0$ and hold its nominal command between solves; evaluate the policy at every environment step. 2. Pseudocode: initialize $u_{\mathrm{mpc}}=0$; for every fast step $k$, observe $z_k,d_k$; if $k\bmod M=0$, roll out $\hat f$, solve the constrained horizon problem, and set $u_{\mathrm{mpc}}$ to its first action; compute $\delta u=\pi_\theta([\operatorname{norm}(z_k),\operatorname{norm}(d_k),u_{\mathrm{mpc}}])$; apply $u_k=\Pi_{\mathcal U(z_k)}(u_{\mathrm{mpc}}+\alpha\delta u)$; store the transition and update the policy with PPO, SAC, or an actor-critic objective. Detach the MPC solution during policy training initially. 3. Computed versus estimated: MPC trajectories, nominal actions, and projections are computed directly; model mismatch, the staleness penalty, and $M_{\mathrm{crit}}$ are estimated from rollouts. 4. First cheap experiment: compare SAC, MPC, and the hybrid controller on Pendulum or a small constrained linear system with injected model mismatch. Sweep $M\in\{1,2,4,8,16\}$ and $\alpha\in\{0.1,0.3,1\}$. Measure return, wall-clock MPC calls, constraint violations, and action power spectra. The quantitative prediction is that MPC calls and MPC wall-clock cost fall approximately as $1/M$, low-frequency action power remains concentrated below the MPC update frequency, and high-frequency power is supplied by the policy. Performance should show a knee at $M_{\mathrm{crit}}$: beyond it, stale nominal actions cause a sharp rise in violations or cost. An initial acceptance criterion is at least 50% fewer MPC calls with no more than 5% return loss and no increase in violation rate before the measured knee.