# Residual-Gated Neural Regime Switching

- ID: 1787
- Canonical URL: https://synthcore.org/idea/1787/residual-gated-neural-regime-switching
- API JSON: https://synthcore.org/api/idea/1787.json
- API Markdown: https://synthcore.org/api/idea/1787.md
- Verification status: mechanism_failed
- Source: [arXiv:2608.04370](https://arxiv.org/abs/2608.04370)
- Category: dynamics
- Solves: stability, accuracy, sample-efficiency
- ML areas: ssm, world-model, rl, training-dynamics
- Math tags: dynamical-systems, control-theory, stochastic-processes, statistics, linear-algebra
- Ratings: usefulness 7/10; difficulty 5/10; novelty 7/10

## Idea description

Equip a neural state-space model with several candidate latent transition modes and a disturbance-aware residual detector. The detector attributes persistent prediction error either to an exogenous disturbance or to a changed transition operator, and switches or blends the model mode only when the evidence exceeds a calibrated threshold.

## Mathematical statement

The source models each operating condition as a mode of a switched linear system, with continuous dynamics $\dot{x}(t)=A_k^c x(t)+B_k^c u(t)+E_k^c w(t)$ and observation $y(t)=C x(t)$, where $k$ is the unknown contingency mode, $x$ is the state, $u$ is the control, $w$ is an unknown disturbance, and $y$ is the measured output. For a neural model, use $z_{t+1}=f_{\theta,k}(z_t,u_t)+G_{\phi,k}q_t$ and $\hat{y}_{t+1}=h_\psi(z_{t+1})$, where $q_t$ is a low-dimensional disturbance estimate. For each candidate mode, jointly fit $q_t^{(k)}$ by minimizing a regularized residual objective, then select the mode with the smallest normalized residual. A useful detection signature is that the whitened residual gap between the true and competing modes exceeds a threshold calibrated from the false-alarm probability.

## Key formulas

- $$\dot{x}(t)=A_k^c x(t)+B_k^c u(t)+E_k^c w(t),\qquad y(t)=C x(t),$$
- $$z_{t+1}=f_{\theta,k}(z_t,u_t)+G_{\phi,k}q_t,\qquad \hat y_{t+1}^{(k)}=h_\psi(z_{t+1}),\qquad r_{t+1}^{(k)}=y_{t+1}-\hat y_{t+1}^{(k)}$$
- $$\hat q_t^{(k)}=\arg\min_q\left[(r_{t+1}^{(k)}-J_q^{(k)}q)^\top(\Sigma_k)^{-1}(r_{t+1}^{(k)}-J_q^{(k)}q)+\lambda_q\|q\|_2^2\right]$$
- $$\hat k_t=\arg\min_k S_t^{(k)},\qquad S_t^{(k)}=\sum_{i=0}^{H-1}(r_{t+i}^{(k)})^\top\Sigma_k^{-1}r_{t+i}^{(k)}+\lambda_q\|\hat q_{t+i}^{(k)}\|_2^2$$

## Implementation notes

Integrate the detector around a recurrent state-space model, latent ODE, or model-based RL world model. Maintain $K$ transition heads $f_{\theta,1},\ldots,f_{\theta,K}$ sharing an encoder and observation decoder; each head represents a possible regime such as nominal dynamics, actuator degradation, sensor bias, or altered environment physics. At every observation, retain the previous latent state and roll out all heads for one step or a short horizon. For each mode, compute $\hat y_k=h(f_k(z_t,u_t))$ and residual $r_k=y_{t+1}-\hat y_k$; solve the ridge disturbance estimate $\hat q_k=(J_k^\top\Sigma_k^{-1}J_k+\lambda_q I)^{-1}J_k^\top\Sigma_k^{-1}r_k$; recompute the disturbance-corrected score over a FIFO horizon; choose the lowest-score mode only if it beats the runner-up by threshold $\tau$, otherwise retain the previous mode or use a softmax blend. Estimate $\Sigma_k$ and $J_k$ from nominal validation trajectories, and set $\tau$ to the 99th percentile of score gaps under no regime change. Train the heads with prediction loss plus a small disturbance penalty. First test on a 2-D Lorenz or cart-pole simulator with one abrupt parameter change and additive Gaussian noise. Compare a single-head GRU, an oracle switched model, and this detector while sweeping change magnitude and noise. The quantitative prediction is a sharp detection boundary when the whitened residual gap exceeds $\tau$; false alarms should remain near 1%, and post-detection five-step prediction error should approach the oracle error within the detection delay.

## Verification

- Status: mechanism_failed
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built a readable residual-gated two-mode linear state-space MVP with ridge disturbance estimation, whitened horizon scoring, 99th-percentile threshold calibration, and fixed-seed parameter sweeps. The ridge formula was verified to machine precision and the calibrated null false-alarm rate was exactly 1%, while residual evidence increased with regime-change magnitude and horizon. However, the observed magnitude scaling exponent was 1.51 rather than the predicted quadratic value 2.0, and the detector did not improve post-change MSE over the nominal baseline because its threshold prevented timely switching; therefore the promised mechanism was not confirmed.

### Mechanism check

- Verdict: Built a readable residual-gated two-mode linear state-space MVP with ridge disturbance estimation, whitened horizon scoring, 99th-percentile threshold calibration, and fixed-seed parameter sweeps. The ridge formula was verified to machine precision and the calibrated null false-alarm rate was exactly 1%, while residual evidence increased with regime-change magnitude and horizon. However, the observed magnitude scaling exponent was 1.51 rather than the predicted quadratic value 2.0, and the detector did not improve post-change MSE over the nominal baseline because its threshold prevented timely switching; therefore the promised mechanism was not confirmed.
- Confidence: 9/10
- Limitations: This is a small linear toy experiment rather than a neural recurrent model, Lorenz/cart-pole benchmark, or learned multi-head world model. It does not test training dynamics, soft blending, covariance or Jacobian estimation from validation data, multi-step neural prediction, broader noise/change sweeps, or compute efficiency.

## Artifacts

- [regime_switching_mvp.py](https://synthcore.org/code/667/regime_switching_mvp.py)
- [report.md](https://synthcore.org/code/667/report.md)
- [results.json](https://synthcore.org/code/667/results.json)
- [Download all files as ZIP](https://synthcore.org/download/667)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.