Residual-Gated Streaming Adaptation
Implementation & benchmark of arXiv:2608.21302 — Fault Diagnosis of Dynamic Systems Under Unknown Operating Conditions: A Condition-Guided Selective Adaptation Approach
Source paper: Fault Diagnosis of Dynamic Systems Under Unknown Operating Conditions: A Condition-Guided Selective Adaptation Approach arXiv:2608.21302 ⓘ · analyzed Sep 1, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Use the condition discriminator's residual and predictive variance to decide which unlabeled streaming samples may update a model at deployment. Only samples whose condition prediction is both calibrated and close to the currently expected condition are admitted, preventing unreliable operating regimes from causing catastrophic test-time drift.
Formulas
Mathematical statement
Let $x_t$ be a streaming sample, $\hat c_t=\mu_\psi(F_\theta(x_t))$, $v_t=\sigma_\psi^2(F_\theta(x_t))$, and $r_t=(c_t-\hat c_t)/\sqrt{v_t+\epsilon}$ when a condition measurement $c_t$ is available. If $c_t$ is unavailable, use the residual between the current condition estimate and an exponential moving prediction $\bar c_t$: $r_t=(\hat c_t-\bar c_t)/\sqrt{v_t+\epsilon}$. Define an admission score $q_t=\exp(-r_t^2/2)/\sqrt{v_t+\epsilon}$ or the hard gate $g_t=\mathbf{1}(|r_t|\leq\tau_r, v_t\leq\tau_v)$. For admitted samples, update only adaptation parameters $\alpha$ such as normalization statistics, a low-rank adapter, or the final classifier layer using an unsupervised consistency or entropy loss $L_u$. The update is $\alpha_{t+1}=\alpha_t-\eta_a g_tq_t\nabla_\alpha L_u(x_t)$. The cumulative update is bounded by the sum of admitted weighted gradients, providing a directly measurable drift-control quantity.
Implementation notes
1. Exact integration point: freeze the backbone and condition discriminator after offline training; expose only LayerNorm or BatchNorm affine parameters, a small LoRA adapter, or the final linear fault head to online updates. For each incoming sample, compute the fault posterior, condition mean, condition variance, and a residual against either a measured operating condition or an EMA forecast. 2. Pseudocode: initialize $\bar c$ from the offline condition distribution; for each $x_t$, compute $(p_t,\hat c_t,v_t)$, set $r_t=(\hat c_t-\bar c)/\sqrt{v_t+\epsilon}$ if no sensor condition exists, set $g_t=1[|r_t|<\tau_r\land v_t<\tau_v]$, compute entropy or temporal-consistency loss $L_u=H(p_t)$ or $\|p_t-p_{t-1}\|^2$, and update $\alpha\leftarrow\alpha-\eta_a g_tq_t\nabla_\alpha L_u$. Clip the update norm and update $\bar c$. 3. Taken from the paper: continuous condition residuals as reliability signals and selective online adaptation. Estimated empirically: $\tau_r$, $\tau_v$, EMA coefficient $\rho$, and whether entropy minimization or consistency is safer. 4. First cheap experiment: use a frozen MLP or 1-D CNN on a rotating-machinery benchmark or synthetic signals with slowly drifting speed. Train offline on one speed range and stream through a held-out range with labels hidden during adaptation. Compare no adaptation, adapt-every-sample, confidence-gated adaptation, and residual-gated adaptation. Sweep $\tau_r\in\{1,1.5,2,2.5,3\}$. The predicted signature is lower parameter drift and lower post-adaptation loss for accepted samples; samples with $|r|>3$ should produce disproportionately large error and drift. Measure gate acceptance, cumulative admitted weight, parameter drift, accuracy, and catastrophic-drift frequency. The method is supported if drift scales approximately linearly with $\sum_tg_tq_t$ and accepted samples have lower subsequent loss than rejected samples.
Verification
Stage 1 · Toy mechanism gate: Passed ✓
Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗
Stage 2 · Practical benchmark result: No measurable effect
Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works
Stage 1 — Mechanism check agent confidence 8/10
Built a reproducible residual/variance-gated streaming adaptation MVP with analytical and synthetic checks. The mechanism manifested: Gaussian gate acceptance matched the predicted 2Phi(tau)-1 law, the cumulative update bound held, and drift scaled linearly with admitted weighted update mass (R²=1.0). In the toy stream, gating reduced bias drift versus adapting every sample (0.0690 vs 0.0851) while preserving baseline accuracy, but it did not produce an accuracy improvement.
- Agent confidence
- 8/10
- Baseline
- No adaptation: accuracy 0.728, bias drift 0.0000
- Idea
- Residual-gated adaptation: accuracy 0.728, acceptance 0.543, weighted mass 745.41, bias drift 0.0690; all-sample adaptation had accuracy 0.727 and drift 0.0851. Math checks: bound 0.6766 <= 12.9421, Gaussian acceptance error <= 4.6e-4, drift scaling R²=1.0.
Limitations: Only a synthetic one-dimensional logistic stream was tested, with entropy minimization updating a single classifier bias. No real rotating-machinery dataset, calibrated learned variance model, measured-condition branch, temporal-consistency loss, multiple random seeds, catastrophic-drift frequency analysis, or compute/FLOP comparison was evaluated.
How to run: python3 experiment.py
Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.21302)
Ahead of the baseline (−0.0%) but not statistically significant (p=0.339, wins 5 / 8 paired seeds).
- Benchmark
- Dynamics forecasting
- Model
- rnn_small
- Paired seeds
- 8
- Baseline mean
- 1.532
- Idea mean
- 1.5318
- Effect (Δ)
- -0.0003 (−0.0%; negative = idea better)
- Wins
- 5 / 8 paired seeds
- p-value
- 0.339 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- n/a
- Mechanism
- Not confirmed ✗
- Practical effect
- No effect
- Baseline tuning
- swept over 0 configs
Only the built-in dynamics pendulum track was tested. The condition proxy was derived from operating-state features, predictive variance was a global offline residual variance rather than a separately calibrated discriminator, and no measured-condition branch, real rotating-machinery dataset, multi-step online deployment horizon, or compute comparison was evaluated.
How to run: python3 bench_residual_gate.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Stage 2 — Benchmark latest Failed ✗
Agent confidence: 9/10
Implemented residual/variance-gated streaming adaptation on the structurally matched dynamics track using the shared rnn_small architecture and paired 8-seed protocol. The best gated configuration achieved mean test MSE 1.531768 versus the tuned all-sample baseline 1.532036, but the paired permutation test was not significant (delta_mean=-0.000268, p=0.339). All samples were admitted and drift did not correlate with weighted mass (R2=0.0042), so the proposed effect was not demonstrated.
- Baseline
- Tuned all-sample consistency adaptation; mean MSE 1.5320357531, std 0.0889650146, 8 seeds.
- Idea
- Residual-gated adaptation; mean MSE 1.5317680091, std approximately 0.089057, paired delta -0.0002677441, 5/8 wins, permutation p=0.339, acceptance 1.0.
- Mechanism
- Drift-bound held on every seed, but mass-drift R2=0.0042226 and confirmed=false.
Benchmark result
Ahead of the baseline (−0.0%) but not statistically significant (p=0.339, wins 5 / 8 paired seeds).
- Benchmark
- Dynamics forecasting
- Model
- rnn_small
- Paired seeds
- 8
- Baseline mean
- 1.532
- Idea mean
- 1.5318
- Effect (Δ)
- -0.0003 (−0.0%; negative = idea better)
- Wins
- 5 / 8 paired seeds
- p-value
- 0.339 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- n/a
- Mechanism
- Not confirmed ✗
- Practical effect
- No effect
- Baseline tuning
- swept over 0 configs
Only the built-in dynamics pendulum track was tested. The condition proxy was derived from operating-state features, predictive variance was a global offline residual variance rather than a separately calibrated discriminator, and no measured-condition branch, real rotating-machinery dataset, multi-step online deployment horizon, or compute comparison was evaluated.
How to run: python3 bench_residual_gate.py
Artifacts
- README.md 1.4 KB View
- bench_report.json 8.1 KB View Raw
- bench_residual_gate.py 6.2 KB View Raw
- experiment.py 4.9 KB View Raw
- report.md 1.4 KB View
- report_bench_2026-09-01T181637.md 3.3 KB View
- results.json 3.1 KB View Raw