# Multimodal Viability Shield

- ID: 532
- Canonical URL: https://synthcore.org/idea/532/multimodal-viability-shield
- API JSON: https://synthcore.org/api/idea/532.json
- API Markdown: https://synthcore.org/api/idea/532.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.03263](https://arxiv.org/abs/2607.03263)
- Category: architecture
- Solves: stability, accuracy, generalization
- ML areas: rl, world-model, training
- Math tags: graph-theory, dynamical-systems, control-theory
- Ratings: usefulness 7/10; difficulty 4/10; novelty 7/10

## Idea description

Attach a finite-state monitor to a neural policy and allow only actions whose successor remains in the simultaneous backward-reachable winning set for all active modes. Modes may encode safety, hardware configuration, and independent task goals. This gives a hard runtime constraint rather than relying on a reward penalty to teach the policy not to enter irreversible dead ends.

## Mathematical statement

Let P=(Q,\\Sigma,\\delta,q_0,Q_m) be a finite monitor, where Q is the monitor-state set, \\Sigma is the event or action alphabet, \\delta(q,\\sigma) is the successor state, q_0 is the initial state, and Q_m is the set of marked goal states. For each mode M_i\\subseteq Q, define W_i^0=M_i\\cap Q_m and W_i^{k+1}=W_i^k\\cup\\{q\\in M_i:\\exists\\sigma\\in\\Sigma,\\delta(q,\\sigma)\\in W_i^k\\}. The fixed point W_i is exactly the set of states that can reach a marked state while never leaving M_i. For r simultaneous modes, use W=\\bigcap_{i=1}^r W_i. At state q, the admissible action set is A(q)=\\{\\sigma:\\delta(q,\\sigma)\\in W\\}. If uncontrollable events \\Sigma_{uc} exist, every enabled uncontrollable successor must remain in W; otherwise q is blocking. The shield uses the paper's modal and multimodal nonblocking construction as a finite viability kernel.

## Key formulas

- $$W_i^0=M_i\cap Q_m,\qquad W_i^{k+1}=W_i^k\cup\{q\in M_i:\exists\sigma\in\Sigma,\;\delta(q,\sigma)\in W_i^k\}.$$
- $$W_i=\operatorname{lfp}\left(X\mapsto(M_i\cap Q_m)\cup\{q\in M_i:\exists\sigma\in\Sigma,\;\delta(q,\sigma)\in X\}\right),\qquad W=\bigcap_{i=1}^r W_i.$$
- $$A(q)=\{\sigma\in\Sigma:\delta(q,\sigma)\in W\},\qquad \pi_{\mathrm{shield}}(\sigma\mid x,q)=\frac{\pi_\theta(\sigma\mid x)\mathbf{1}[\sigma\in A(q)]}{\sum_{a\in A(q)}\pi_\theta(a\mid x)}.$$
- $$\forall\sigma\in\Sigma_{uc}\text{ enabled at }q:\quad\delta(q,\sigma)\in W.$$

## Implementation notes

(1) Integrate the shield immediately before action sampling in a discrete-action PPO or SAC policy. Maintain a monitor state q for every environment instance and update it after each selected action or observed event using a deterministic transition table delta. Define one mode M_i for each requirement, such as collision-free states, valid hardware configuration, or progress toward a particular subgoal. Mark successful terminal or intermediate goal states in Q_m. (2) Before training, compute every W_i with reverse graph search. Initialize a boolean array with M_i intersect Q_m, enqueue those states, and repeatedly add a predecessor q when q is in M_i and has at least one enabled successor already in the queue. Stop when no state is added, intersect all arrays to obtain W, and precompute A(q) by scanning action successors. At runtime set policy logits outside A(q) to negative infinity before sampling. If A(q) is empty, record a blocking event and use an explicitly defined recovery action or terminate the episode; do not silently ignore the violation. (3) The fixed point and action mask are exact for the symbolic monitor. Empirically estimate only abstraction errors by checking whether simulator transitions disagree with delta; uncertain successors should be conservatively masked. (4) First test on MiniGrid or a 10x10 gridworld with hazards, locked-room configurations, and two required goals. Compare unconstrained PPO, reward-penalty PPO, and shielded PPO at equal environment steps. Measure total-goal success, irreversible dead-end frequency, constraint violations, return, and wall-clock overhead. The expected signal is near-zero violations and higher multi-goal success with only small action-filtering cost.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built a finite-state multimodal viability shield with least-fixed-point winning-set computation, rank tracking, action masks, and explicit blocking behavior. All tested mechanism predictions were confirmed: fixed-point rank matched shortest safe-path distance across 8 graph lengths, maximum rank followed the derived floor((L+2)/2) scaling, adding a mode monotonically shrank winning sets and masks, and shielded rollouts had zero violations. In the toy benchmark, shielding improved success from 25.98% to 59.42% and reduced violations from 74.02% to 0%, at the cost of substantially more steps under random masked action selection.

### Mechanism check

- Verdict: Built a finite-state multimodal viability shield with least-fixed-point winning-set computation, rank tracking, action masks, and explicit blocking behavior. All tested mechanism predictions were confirmed: fixed-point rank matched shortest safe-path distance across 8 graph lengths, maximum rank followed the derived floor((L+2)/2) scaling, adding a mode monotonically shrank winning sets and masks, and shielded rollouts had zero violations. In the toy benchmark, shielding improved success from 25.98% to 59.42% and reduced violations from 74.02% to 0%, at the cost of substantially more steps under random masked action selection.
- Confidence: 9/10
- Limitations: Tested only a deterministic discrete toy transition system, not PPO/SAC, MiniGrid, continuous actions, simulator/model-transition disagreement, or enabled uncontrollable events. The runtime comparison used uniformly random action selection rather than a learned neural policy and did not measure wall-clock masking overhead.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/249/bench_report.json)
- [bench_shield.py](https://synthcore.org/code/249/bench_shield.py)
- [report.md](https://synthcore.org/code/249/report.md)
- [report_bench_2026-08-30T213809.md](https://synthcore.org/code/249/report_bench_2026-08-30T213809.md)
- [run_experiment.py](https://synthcore.org/code/249/run_experiment.py)
- [viability_shield.py](https://synthcore.org/code/249/viability_shield.py)
- [Download all files as ZIP](https://synthcore.org/download/249)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.