# Feasible Action Mapping Safety Layer

- ID: 1341
- Canonical URL: https://synthcore.org/idea/1341/feasible-action-mapping-safety-layer
- API JSON: https://synthcore.org/api/idea/1341.json
- API Markdown: https://synthcore.org/api/idea/1341.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.23930](https://arxiv.org/abs/2607.23930)
- Category: architecture
- Solves: stability, accuracy, sample-efficiency
- ML areas: rl, optimizer, training-dynamics
- Math tags: control-theory, optimization, dynamical-systems, geometry
- Ratings: usefulness 7/10; difficulty 6/10; novelty 5/10

## Idea description

Let a neural policy emit an unconstrained abstract action z, then solve a state-dependent feasibility problem that maps z to an admissible optimal-control parameter p before execution. Unlike coordinate-wise clipping, the mapping accounts for predicted dynamics, coupled state and input constraints, and recursive feasibility, allowing the policy to retain a simple unconstrained output space while the controller enforces plant constraints.

## Mathematical statement

Let x_t be the observed state, z_t = pi_theta(x_t) in R^d the neural policy output, and p be a control parameter or finite-horizon sequence. Let x_{k+1} = f(x_k,u_k), with u_k = Phi_k(x_t,p), horizon H, state constraint x_k in X, input constraint u_k in U, and terminal safe set X_f. The feasible abstract-action set consists of outputs for which some mapped parameter produces a constraint-satisfying trajectory ending in X_f. The safety layer computes the weighted projection p_t^* minimizing the distance from z_t subject to the OCP constraints, then executes Phi_0(x_t,p_t^*). If the model is exact, the optimization remains feasible, and X_f is control invariant, recursive feasibility implies constraint satisfaction at future steps. The measurable signature is zero violations in the certified regime and a sharp increase in projection distance as policy outputs cross the feasible-set boundary.

## Key formulas

- $$x_{k+1}=f(x_k,u_k),\qquad u_k=\Phi_k(x_t,p),\qquad k=0,\ldots,H-1,$$
- $$\mathcal{Z}_{f}(x_t)=\left\{z:\exists p,\ x_0=x_t,\ x_{k+1}=f(x_k,\Phi_k(x_t,p)),\ x_k\in\mathcal{X},\ \Phi_k(x_t,p)\in\mathcal{U},\ x_H\in\mathcal{X}_f\right\},$$
- $$p_t^*=\arg\min_{p}\frac{1}{2}\|p-z_t\|_{W}^{2}\quad\text{s.t.}\quad x_{k+1}=f(x_k,\Phi_k(x_t,p)),\ x_k\in\mathcal{X},\ \Phi_k(x_t,p)\in\mathcal{U},\ x_H\in\mathcal{X}_f,$$
- $$a_t=\Phi_0(x_t,p_t^*),\qquad d_t=\|p_t^*-z_t\|_{W};\quad z_t\in\mathcal{Z}_f(x_t)\Rightarrow d_t=0.$$

## Implementation notes

1. Integration point: place the layer after the policy network and before the environment or actuator. Begin with a low-dimensional control parameter p, such as constant acceleration, spline coefficients, or an H-step action sequence. The policy outputs z_t, the safety layer solves the constrained finite-horizon problem, and only the first action is sent to the simulator. Use OSQP for linear or quadratic models and IPOPT or CasADi for nonlinear models. Initially stop gradients through the optimizer; later use KKT sensitivity or a differentiable QP layer. 2. Pseudocode: compute z = policy(x); initialize p = z; set x[0] = x; for k in 0..H-1 compute u[k] = Phi_k(x,p), predict x[k+1] = f(x[k],u[k]); solve the weighted projection problem subject to state, input, and terminal constraints; execute u[0]; log p*, solver status, and d = ||p*-z||_W; update the policy using the RL reward. If infeasible, use a preverified backup controller and record the event rather than silently relaxing hard constraints. 3. The control construction computes the feasible trajectory, projection, first action, and terminal-set certificate. Empirically estimate model mismatch, solver failure rates, and the geometry of the feasible set. 4. First experiment: compare PPO with action clipping against PPO plus this layer on cart-pole or a bounded point-mass task with position and acceleration limits. Sweep policy-output scale and measure reward, violation rate, projection distance, and runtime. The prediction is near-zero violations whenever the solver is feasible, d approximately zero for outputs inside the feasible region, and a sharp rise in mean d when output scale crosses the feasibility boundary. For a linear fixed-horizon system, sample the feasible set offline and require online feasibility classification to agree at least 95 percent of the time.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built an exact finite-horizon predictive safety layer that projects an unconstrained scalar action onto the state-dependent feasible interval for a bounded point-mass system. The mechanism manifested: feasible outputs had maximum projection distance 0, outside-distance slopes were -1/+1 with error 3.3e-16, the velocity-dependent boundary shift matched theory within 5.6e-17, and all 10,000 feasible random projections were certified. In the stress test clipping had 84 one-step violations while predictive projection had 0; the toy controller also achieved mean reward -51.20 versus -52.77 for clipping, though this is not a broad RL result.

### Mechanism check

- Verdict: Built an exact finite-horizon predictive safety layer that projects an unconstrained scalar action onto the state-dependent feasible interval for a bounded point-mass system. The mechanism manifested: feasible outputs had maximum projection distance 0, outside-distance slopes were -1/+1 with error 3.3e-16, the velocity-dependent boundary shift matched theory within 5.6e-17, and all 10,000 feasible random projections were certified. In the stress test clipping had 84 one-step violations while predictive projection had 0; the toy controller also achieved mean reward -51.20 versus -52.77 for clipping, though this is not a broad RL result.
- Confidence: 9/10
- Limitations: This is an exact-model scalar constant-acceleration toy system, not PPO, a neural policy training experiment, or a nonlinear environment. Recursive feasibility across replanning steps, solver failures, model mismatch, differentiability, computational timing, and backup-controller behavior were not tested; the clipping closed-loop scenario happened not to produce violations despite the random stress test exposing its weakness.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/526/bench_report.json)
- [report.md](https://synthcore.org/code/526/report.md)
- [report_bench_2026-08-31T165747.md](https://synthcore.org/code/526/report_bench_2026-08-31T165747.md)
- [results.json](https://synthcore.org/code/526/results.json)
- [safety_layer_experiment.py](https://synthcore.org/code/526/safety_layer_experiment.py)
- [stage2_bench.py](https://synthcore.org/code/526/stage2_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/526)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.