# IMM Stale-Feedback Detector

- ID: 2545
- Canonical URL: https://synthcore.org/idea/2545/imm-stale-feedback-detector
- API JSON: https://synthcore.org/api/idea/2545.json
- API Markdown: https://synthcore.org/api/idea/2545.md
- Verification status: failed_benchmark
- Source: [arXiv:2608.18824](https://arxiv.org/abs/2608.18824)
- Category: dynamics
- Solves: stability, scalability
- ML areas: optimizer, rnn, ssm, training-dynamics
- Math tags: probability, stochastic-processes, control-theory, dynamical-systems, statistics
- Ratings: usefulness 7/10; difficulty 5/10; novelty 7/10

## Idea description

Add an interacting-multiple-model monitor to a recurrent or distributed neural training loop, with one state estimator for each candidate feedback delay. The monitor detects when gradients, hidden-state feedback, or parameter acknowledgements become stale, allowing the system to reduce the learning rate, discard delayed updates, or switch to a safe synchronous mode before delayed feedback destabilizes training.

## Mathematical statement

The paper models a known linear state process with delayed observations, $x_{t+1}=Ax_t+Bu_t+w_t$ and $y_t=Cx_{t-d_t}+\nu_{t-d_t}$, where $d_t$ is an unknown delay mode. For neural training, let $z_t$ be a low-dimensional monitored state such as loss, gradient norm, parameter norm, and validation residual; use a local linearization $z_{t+1}=A_tz_t+B_tu_t+w_t$ and candidate delays $d_i\in\{0,\ldots,D\}$. Each mode $i$ maintains $(\hat z_t^i,P_t^i)$ and predicts the observation with a delay-specific observation operator $H_i$. Given Markov mode transitions $T_{ji}=P(d_t=i\mid d_{t-1}=j)$, IMM mixing uses $\mu_{j\mid i,t-1}=T_{ji}\pi_{t-1}^j/\sum_kT_{ki}\pi_{t-1}^k$. The mode likelihood is the Gaussian innovation density $\ell_t^i=\mathcal N(r_t^i;0,S_t^i)$, where $r_t^i=y_t-H_i\hat z_{t\mid t-1}^i$ and $S_t^i=H_iP_{t\mid t-1}^iH_i^\top+R$. The posterior is $\pi_t^i=\ell_t^i\sum_jT_{ji}\pi_{t-1}^j/\sum_k\ell_t^k\sum_jT_{jk}\pi_{t-1}^j$. The alarm is raised when $\pi_t^0\leq\xi$, with mode zero denoting no delay. Under a persistent delay, the expected log posterior odds grow at approximately the per-step KL divergence between the true and no-delay innovation distributions.

## Key formulas

- $$z_{t+1}=A_tz_t+B_tu_t+w_t,\qquad y_t=H_{d_t}z_{t-d_t}+\nu_t,\qquad w_t\sim\mathcal N(0,Q),\ \nu_t\sim\mathcal N(0,R).$$
- $$\mu_{j\mid i,t-1}=\frac{T_{ji}\pi_{t-1}^{j}}{\sum_{k=0}^{D}T_{ki}\pi_{t-1}^{k}},\qquad \bar z_{t-1}^{i}=\sum_{j=0}^{D}\mu_{j\mid i,t-1}\hat z_{t-1}^{j}.$$
- $$r_t^i=y_t-H_i\hat z_{t\mid t-1}^{i},\quad S_t^i=H_iP_{t\mid t-1}^{i}H_i^\top+R,\quad \ell_t^i=\frac{\exp[-\tfrac12(r_t^i)^\top(S_t^i)^{-1}r_t^i]}{(2\pi)^{m/2}|S_t^i|^{1/2}}.$$
- $$\pi_t^i=\frac{\ell_t^i\sum_jT_{ji}\pi_{t-1}^{j}}{\sum_k\ell_t^k\sum_jT_{jk}\pi_{t-1}^{j}},\qquad \textsc{alarm}(t)=\mathbf 1[\pi_t^0\leq\xi].$$

## Implementation notes

1. Integration point: begin with a small GRU or a distributed MLP trained by one asynchronous worker. Define $z_t\in\mathbb R^p$ as a monitored state containing current loss, exponentially smoothed loss, gradient norm, parameter norm, update norm, and worker-to-server residual. Keep the optimizer unchanged and run the IMM monitor after every optimizer step. Candidate modes are delays $d_i=0,1,\ldots,D$, with $D=4$ for the MVP. For a nonlinear network, estimate $A_t$ using a diagonal or low-rank local linear model fitted online from the last 100 monitor states, or use a constant autoregressive matrix learned during clean warmup. 2. Pseudocode: initialize $\pi^i=1/(D+1)$, $\hat z^i=z_0$, and $P^i=I$. At each step, compute $\mu_{j|i}$, mix means and covariances, propagate each mode with $\hat z^i\leftarrow A_t\bar z^i+B_tu_t$ and $P^i\leftarrow A_t\bar P^iA_t^\top+Q$, construct the delay-specific predicted observation using the corresponding ring-buffer entry $\hat z_{t-d_i}^i$, calculate $r_t^i,S_t^i,\ell_t^i$, normalize the posterior $\pi_t^i$, and update each EKF covariance. If $\pi_t^0\leq\xi$ for three consecutive steps, reduce the learning rate by 4x and reject updates older than the median detected delay. 3. Computed versus estimated: IMM equations and posterior normalization are exact; $A_t,Q,R$ and transition matrix $T$ are estimated from a clean calibration run. Set $R$ to the empirical covariance of monitor residuals and $Q$ to 0.01--0.1 times that covariance. Estimate $\xi$ from the 99.5th percentile of the minimum clean no-delay posterior over calibration windows. 4. First cheap experiment: train a two-layer MLP on MNIST with one asynchronous worker, then inject a fixed delay of $d=2$ steps into worker gradients at step 2,000. Compare SGD, SGD with gradient-age clipping, and SGD plus IMM. Record posterior trajectories, detection delay, false alarms, and loss spikes. The quantitative prediction is that the posterior for mode $d=2$ will dominate after the attack and that expected log odds $\log(\pi_t^2/\pi_t^0)$ will have an approximately linear slope equal to the empirical innovation KL divergence. Measure the clean alarm rate below 0.5%, and verify that a delay of two steps is detected within 20--50 updates when its innovation KL divergence is at least 0.1 nats per step. Sweep injected delay and learning rate; the learned stability boundary should shift to larger tolerable learning rates when delayed updates are rejected.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: no_effect
- Verdict: Built an IMM stale-feedback detector for candidate delays 0..4 using exact Gaussian innovation likelihoods, Markov mode mixing, posterior updates, and alarms. The mechanism manifested: posterior normalization error was 4.4e-16, persistent delay-2 reached median posterior 0.96 versus 0.006 for no-delay, and cumulative innovation log-likelihood slope divided by empirical KL was 0.99-1.01 across noise sweeps. Detection time increased from 6 to 23 steps as KL decreased, but the practical control benchmark produced no alarm and no improvement over baseline, so this supports the detector math rather than a demonstrated neural-training win.

### Mechanism check

- Verdict: Built an IMM stale-feedback detector for candidate delays 0..4 using exact Gaussian innovation likelihoods, Markov mode mixing, posterior updates, and alarms. The mechanism manifested: posterior normalization error was 4.4e-16, persistent delay-2 reached median posterior 0.96 versus 0.006 for no-delay, and cumulative innovation log-likelihood slope divided by empirical KL was 0.99-1.01 across noise sweeps. Detection time increased from 6 to 23 steps as KL decreased, but the practical control benchmark produced no alarm and no improvement over baseline, so this supports the detector math rather than a demonstrated neural-training win.
- Confidence: 8/10
- Limitations: No MLP, GRU, asynchronous worker, gradient-age rejection, or GPU training experiment was completed. The successful verification assumes a known scalar linear process and known delayed state history; online estimation of A, Q, R and IMM performance under neural monitor noise were not tested. The control benchmark did not trigger an alarm, so learning-rate mitigation remains unvalidated.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 1
- Paired wins: 0/8
- Benchmark verdict: no measurable effect

## Artifacts

- [bench_imm.py](https://synthcore.org/code/977/bench_imm.py)
- [bench_report.json](https://synthcore.org/code/977/bench_report.json)
- [control_results.json](https://synthcore.org/code/977/control_results.json)
- [imm_experiment.py](https://synthcore.org/code/977/imm_experiment.py)
- [mini_control.py](https://synthcore.org/code/977/mini_control.py)
- [nn_signature.py](https://synthcore.org/code/977/nn_signature.py)
- [report.md](https://synthcore.org/code/977/report.md)
- [report_bench_2026-09-01T172034.md](https://synthcore.org/code/977/report_bench_2026-09-01T172034.md)
- [results.json](https://synthcore.org/code/977/results.json)
- [Download all files as ZIP](https://synthcore.org/download/977)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.