Heteroscedastic Condition-Adversarial Representation
Implementation & benchmark of arXiv:2608.21302 — Fault Diagnosis of Dynamic Systems Under Unknown Operating Conditions: A Condition-Guided Selective Adaptation Approach
Source paper: Fault Diagnosis of Dynamic Systems Under Unknown Operating Conditions: A Condition-Guided Selective Adaptation Approach arXiv:2608.21302 ⓘ · analyzed Sep 1, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Attach a Gaussian condition discriminator to an intermediate neural representation and train it adversarially against the fault classifier. The discriminator predicts both the mean and uncertainty of a continuous operating condition, forcing the encoder to remove condition-dependent variation without treating the condition as a small set of artificial domains.
Formulas
Mathematical statement
Let $x$ be an input, $y$ its fault label, $c\in\mathbb{R}$ its continuous operating condition, $F_\theta(x)$ the encoder representation, and $G_\phi$ the fault classifier. A condition discriminator $D_\psi$ outputs a mean $\mu_\psi(F_\theta(x))$ and positive variance $\sigma_\psi^2(F_\theta(x))=\operatorname{softplus}(s_\psi(F_\theta(x)))+\epsilon$. Its heteroscedastic Gaussian negative log-likelihood is $L_d=\mathbb{E}[\frac12\log\sigma_\psi^2+\frac{(c-\mu_\psi)^2}{2\sigma_\psi^2}]$. The discriminator minimizes $L_d$, while the encoder maximizes it, and the classifier minimizes cross-entropy $L_{cls}$. The saddle objective is $\min_{\theta,\phi}\max_{\psi}\;L_{cls}(\theta,\phi)-\lambda L_d(\theta,\psi)$, implemented by a gradient-reversal layer with coefficient $\lambda$. The useful property is that the encoder is penalized when condition information remains predictable, while the variance branch models noisy or ambiguous condition labels.
Implementation notes
1. Exact integration point: choose the penultimate feature tensor of a small MLP, 1-D CNN, Transformer encoder, or state-space model. Add two linear heads on pooled features: one outputs $\mu$ and one outputs $s$, with variance $\sigma^2=\operatorname{softplus}(s)+10^{-6}$. Add a gradient-reversal layer between the feature and condition heads. Train the fault head with cross-entropy and the condition head with the Gaussian negative log-likelihood. 2. Pseudocode: for each minibatch $(x,y,c)$, compute $h=F_\theta(x)$, $p=G_\phi(h)$, $(\mu,s)=D_\psi(\operatorname{GRL}_\lambda(h))$, $v=\operatorname{softplus}(s)+\epsilon$, then $L=-\operatorname{mean}\log p_y+\lambda\operatorname{mean}[0.5\log v+0.5(c-\mu)^2/v]$ and backpropagate once. Warm up with $\lambda=0$ for 5 epochs, then ramp to a target value. 3. Computed from the paper: the Gaussian condition objective, adversarial sign, and mean/variance parameterization. Estimated empirically: the best $\lambda$, feature location, and condition normalization. 4. First cheap experiment: train a 2-layer 1-D CNN on a gearbox or synthetic rotating-signal dataset with fault labels and a continuous speed/load variable. Compare ERM, discrete domain-adversarial training, and this method for $\lambda\in\{0,0.01,0.03,0.1,0.3,1\}$. Hold out operating-condition intervals rather than random samples. The predicted signature is that condition-probe $R^2$ decreases with $\lambda$, while held-out-condition fault accuracy first improves and then collapses beyond an over-adaptation region. Report the crossover and test whether the best $\lambda$ occurs before validation fault loss increases by 10%.
Verification
Stage 1 · Toy mechanism gate: Failed ✗
Stage 2 · Mechanism transferred to benchmark: Not tested
Stage 2 · Practical benchmark result: Not run
Methodology: Toy-system gate first; the benchmark stage runs only after a pass. How verification works
Stage 1 — Mechanism check agent confidence 8/10
Built and ran a small heteroscedastic condition-adversarial MLP with GRL, Gaussian mean/variance heads, held-out operating-condition evaluation, and quantitative math checks. The Gaussian NLL optimum matched the residual variance (1.00074 vs 1.00073), and measured GRL gradient norms scaled exactly linearly with lambda (0, 0.1, 0.3, 1.0 gave 0, 0.08246, 0.24739, 0.82462). However, the sweep did not show a reliable monotonic reduction in independent condition leakage or a stable fault-accuracy crossover; accuracy varied only slightly and the strongest result at lambda=1.0 was not sufficient evidence of a robust win, so the mechanism-level success criterion is not met.
- Agent confidence
- 8/10
- Baseline
- lambda=0: held-out accuracy 0.955, independent condition-probe R2 -2.873, condition-head NLL 0.2213
- Idea
- Best observed lambda=1.0: held-out accuracy 0.980, independent condition-probe R2 -5.878, condition-head NLL 0.4221; lambda=0.3 had accuracy 0.955 and head NLL 0.4266
Limitations: Only one fixed synthetic dataset seed and one small MLP were tested; no repeated seeds, discrete domain-adversarial baseline, CNN or real rotating-signal data, FLOP/speed comparison, calibration metrics, or statistical significance analysis was performed. The held-out interval induces extrapolation, making negative linear-probe R2 values difficult to interpret as a clean leakage measure.
How to run: python3 experiment.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Artifacts
- experiment.py 4.3 KB View Raw
- report.md 1.6 KB View
- results.json 1.2 KB View Raw