# Bernstein-Safe Policy Head

- ID: 2919
- Canonical URL: https://synthcore.org/idea/2919/bernstein-safe-policy-head
- API JSON: https://synthcore.org/api/idea/2919.json
- API Markdown: https://synthcore.org/api/idea/2919.md
- Verification status: unverified
- Source: [arXiv:2607.23040](https://arxiv.org/abs/2607.23040)
- Category: architecture
- Solves: stability, accuracy, sample-efficiency
- ML areas: rl, world-model, mlp, training
- Math tags: control-theory, optimization, dynamical-systems, approximation-theory
- Ratings: usefulness 6/10; difficulty 5/10; novelty 7/10

## Idea description

Replace a neural controller's pointwise action outputs over a finite horizon with Bernstein control points whose convex hull satisfies actuator and trajectory constraints. The network predicts the control points, while a robust margin accounts for bounded tracking or model-prediction error, making continuous-time actuator feasibility checkable from finitely many inequalities.

## Mathematical statement

For horizon $t\in[0,T]$, define the degree-$n$ Bernstein basis $B_{k,n}(s)=\binom{n}{k}s^k(1-s)^{n-k}$ with normalized time $s=t/T$ and control trajectory $u(t)=\sum_{k=0}^{n}c_kB_{k,n}(s)$, where $c_k\in\mathbb{R}^m$ are neural-network outputs. Since $B_{k,n}(s)\geq0$ and $\sum_kB_{k,n}(s)=1$, every $u(t)$ lies in the convex hull of the control points; therefore componentwise bounds $u_{\min}\leq c_k\leq u_{\max}$ imply $u_{\min}\leq u(t)\leq u_{\max}$ for all $t$. Derivative control points are $d_k=n(c_{k+1}-c_k)/T$, giving $\dot u(t)=\sum_{k=0}^{n-1}d_kB_{k,n-1}(s)$, so slew-rate limits follow from $d_{\min}\leq d_k\leq d_{\max}$. If execution has additive tracking error $e_u(t)$ satisfying $\|e_u(t)\|_\infty\leq\varepsilon_u$, robust actuator feasibility is guaranteed by $u_{\min}+\varepsilon_u\leq c_k\leq u_{\max}-\varepsilon_u$; more generally, for a feedback correction $K e(t)$ with $\|e(t)\|_\infty\leq\varepsilon_e$, use margin $\varepsilon_u=\|K\|_\infty\varepsilon_e$.

## Key formulas

- $$B_{k,n}(s)=\binom{n}{k}s^k(1-s)^{n-k},\qquad u(t)=\sum_{k=0}^{n}c_kB_{k,n}(t/T),\qquad B_{k,n}(s)\geq0,\quad\sum_{k=0}^{n}B_{k,n}(s)=1.$$
- $$u_{\min}+\varepsilon_u\mathbf{1}\leq c_k\leq u_{\max}-\varepsilon_u\mathbf{1}\quad\forall k\ \Longrightarrow\ u_{\min}\leq u(t)+e_u(t)\leq u_{\max}\quad\forall t,$$
- $$d_k=\frac{n}{T}(c_{k+1}-c_k),\qquad \dot u(t)=\sum_{k=0}^{n-1}d_kB_{k,n-1}(t/T),\qquad d_{\min}\leq d_k\leq d_{\max}.$$
- $$\varepsilon_u=\|K\|_\infty\varepsilon_e,\qquad \|K\|_\infty=\max_i\sum_j|K_{ij}|.$$

## Implementation notes

1. Integration point: use an MLP policy head that receives the current observation and predicts $m(n+1)$ control-point values instead of one action. For a receding-horizon controller, execute only the first action or first short segment, then re-predict at the next observation; for offline trajectory generation, decode the complete Bernstein curve. 2. Pseudocode: compute raw control points $C_{raw}=\mathrm{policy}_\theta(o)$; estimate or provide an error bound $\varepsilon_e$; set $\varepsilon_u=\|K\|_\infty\varepsilon_e$; project $C_{raw}$ onto $u_{min}+\varepsilon_u\leq C\leq u_{max}-\varepsilon_u$; compute $D=n(C_{1:}-C_{:-1})/T$ and either project onto slew constraints or add a hinge penalty; evaluate $u(t_j)=B(t_j)C$ on the simulator grid. 3. Computed from the mechanism: Bernstein basis values, control-point bounds, derivative control points, and the convex-hull certificate. Estimated empirically: $\varepsilon_e$ from a held-out rollout quantile or a conservative disturbance bound, and $K$ from the tracking controller. 4. First experiment: train PPO or model-predictive imitation on a 2D point-mass or pendulum with identical network size, horizon, and environment steps. Compare a standard action head, an unconstrained Bernstein head, and the robust Bernstein head; ablate the error margin and slew constraints. 5. Pre-registered signature: with all control-point margins satisfied, the maximum actuator violation over at least 1000 uniformly sampled times per rollout must be zero up to numerical tolerance $10^{-6}$, including between training-grid points. The measured violation rate must stay below 0.1%; otherwise reject the convex-hull transfer. Removing margin $\varepsilon_u$ should produce violations once observed tracking error exceeds the removed margin, while the robust head should remain feasible. If violations occur despite valid control-point inequalities, reject the implementation or the assumed actuator/error model.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.