△ Mechanism confirmed, baseline not beaten 2026

Koopman Hankel Dual Autoencoder

Implementation & benchmark of arXiv:2608.17698 — Fault detection on manifolds of nonlinear dynamical systems with dual autoencoders

Usefulness7/10

Difficulty5/10

Novelty6/10

Architecture Stability Accuracy Generalization Rnn Ssm Transformer Regularization Training dynamics Dynamical systems Control theory Linear algebra Statistics

Source paper: Fault detection on manifolds of nonlinear dynamical systems with dual autoencoders arXiv:2608.17698 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace pointwise sequence reconstruction with reconstruction of overlapping past and future Hankel windows in a shared latent manifold. A first encoder compresses the delay-coordinate trajectory, while a second decoder or predictor reconstructs the future block from the latent state; training therefore penalizes representations that fit observations but do not preserve dynamical evolution.

Formulas

$$\mathcal{Y}^{-}=\begin{bmatrix}y_0&y_1&\cdots&y_{N-1}\\y_1&y_2&\cdots&y_N\\\vdots&\vdots&\ddots&\vdots\\y_{p-1}&y_p&\cdots&y_{N+p-2}\end{bmatrix},\qquad \mathcal{Y}^{+}=\begin{bmatrix}y_p&y_{p+1}&\cdots&y_{N+p-1}\\y_{p+1}&y_{p+2}&\cdots&y_{N+p}\\\vdots&\vdots&\ddots&\vdots\\y_{2p-1}&y_{2p}&\cdots&y_{N+2p-2}\end{bmatrix}.$$

$$\mathcal{Y}^{+}\approx K\mathcal{Y}^{-},\qquad K=\arg\min_{K}\left\|\mathcal{Y}^{+}-K\mathcal{Y}^{-}\right\|_{F}^{2}+\lambda_K\|K\|_{F}^{2},$$

$$\mathcal{L}=\|\mathcal{Y}^{-}-D^-_\psi(E_\phi(\mathcal{Y}^{-}))\|_{F}^{2}+\alpha\|\mathcal{Y}^{+}-D^+_\omega(E_\phi(\mathcal{Y}^{-}))\|_{F}^{2}+\beta\|E_\phi(\mathcal{Y}^{+})-A E_\phi(\mathcal{Y}^{-})\|_{2}^{2}+\gamma\|A\|_{2}^{2},$$

$$T=(e-\mu)^\top\Sigma^{-1}(e-\mu),\qquad e=\operatorname{vec}\!\left(\mathcal{Y}^{+}-D^+_\omega(E_\phi(\mathcal{Y}^{-}))\right).$$

Mathematical statement

For an observed sequence y_k in R^r, optionally lift each sample with observables epsilon_n(y_k) in R^q, where n indexes the chosen observable set. With delay depth p and window count N, define the past and future Hankel matrices Y^- and Y^+ from consecutive samples; the lifted versions replace y_t by epsilon_n(y_t). A finite Koopman approximation assumes Y^+ is approximately K Y^-, where K is a learned linear evolution operator in lifted coordinates. Train an encoder z=E_phi(Y^-), a past decoder Yhat^-=D^-_psi(z), and a future decoder or predictor Yhat^+=D^+_omega(z), with an optional latent transition z_next=A z. The useful property is temporal consistency: low past reconstruction error alone is insufficient unless the same latent neighborhood also gives low future reconstruction error. The latent spectral radius rho(A) supplies a directly measurable bounded-rollout condition.

Implementation notes

1. Integration point: use this module before an RNN, state-space model, or transformer for windowed sequence representation, or use it as a standalone anomaly detector. Choose delay depth p, construct overlapping windows of length 2p, and arrange each window as a p-by-r past block and a p-by-r future block. Implement E_phi as an MLP, temporal convolution, or transformer over the Hankel block. Implement D^- and D^+ as decoders. Optionally parameterize A as a stable latent transition by computing its eigendecomposition after each update and rescaling it when rho(A) exceeds a target value below one.

2. Pseudocode: for every nominal batch, form Yminus and Yplus; compute z=E(Yminus), past_hat=Dminus(z), and future_hat=Dplus(z); minimize the displayed loss, initially using alpha=1 and beta between 0.1 and 1. After training, collect future residual vectors e on held-out nominal windows and estimate their mean mu and covariance Sigma with shrinkage, Sigma=(1-rho)Sigma_emp+rho I. At test time compute T and flag a window when T exceeds the empirical 0.99 or 0.995 nominal quantile. For forecasting, use A^j z or a learned recurrent rollout and decode each future state.

3. Computed versus estimated: Hankel construction and loss evaluation are deterministic. K or A is estimated by ridge regression or gradient descent, while mu and Sigma are estimated from nominal validation residuals. Report the normalized Koopman residual ||Yplus-K Yminus||_F/||Yplus||_F, future reconstruction error, and latent spectral radius rho(A).

4. First cheap experiment: simulate Lorenz-63 with Gaussian observation noise, train on nominal trajectories, and inject a parameter fault by changing a Lorenz coefficient. Compare a pointwise autoencoder, a delay-window autoencoder, and this dual model at equal parameter count. The predicted signature is an initial decrease in Koopman residual as p increases, followed by a plateau from redundant delays. Long-horizon latent rollouts should remain bounded when rho(A)<1 and diverge or grow rapidly near rho(A)=1. Measure whether the empirical boundedness boundary agrees with rho(A)=1 within 20 percent, and whether fault windows cross the calibrated 0.99 Mahalanobis threshold more often than nominal windows.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a reproducible Koopman Hankel dual autoencoder MVP with Hankel construction, ridge Koopman estimation, past/future reconstruction, and CUDA-safe training. The mechanism manifested: noise residual scaled linearly with sigma (R²=0.99985), rollout growth changed sign exactly at |A|=1, and delay depth p=2 reduced the AR(2) Koopman residual from 0.9992 at p=1 to 8.47e-9. In the small fault experiment, the dual future-error detector achieved AUC 0.6967 versus 0.6562 for the past-reconstruction baseline, a modest positive signal.

Agent confidence: 8/10

Baseline: Past-only autoencoder fault AUC=0.6562; nominal past MSE=0.000325; fault past MSE=0.004017.
Idea: Dual past+future autoencoder future-error fault AUC=0.6967; nominal future MSE=0.000996; fault future MSE=0.007727. Math: noise residual slope=0.9845 with R²=0.99985; stability boundary observed/predicted=1.00/1.00; p=1 versus p=2 residual=0.9992 versus 8.47e-9.

Limitations: The toy system was scalar AR(2), not Lorenz-63. The experiment was small and used 700 optimization steps. The baseline was evaluated with past reconstruction while the dual model was evaluated with future reconstruction, so the AUC comparison is indicative rather than fully matched. Learned long-horizon latent rollouts and Mahalanobis threshold calibration were not tested.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.17698) Sep 2, 2026

Ahead of the baseline (−7.2%) but not statistically significant (p=0.3691, wins 6 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0007
Idea mean: 0.0006
Effect (Δ): -0 (−7.2%; negative = idea better)
Wins: 6 / 8 paired seeds
p-value: 0.3691 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±17.6%
Mechanism: Confirmed ✓
Practical effect: Inconclusive
Baseline tuning: swept over 6 configs

Limitations:

Only the registered built-in dynamics track and rnn_small architecture were tested. The run used 400 training samples, 100 test samples, 15 epochs, and a small hyperparameter sweep; Lorenz-63 transfer, long-horizon rollout quality, anomaly detection, and broader delay-depth studies were not tested.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Failed ✗

Agent confidence: 9/10

The Koopman Hankel dual autoencoder was implemented as a local training intervention for the registered dynamics/rnn_small benchmark. It showed a modest lower test MSE than the tuned baseline, but the paired permutation test was nonsignificant (p=0.36915), so the harness verdict is no significant win. The trained-model mechanism signature was confirmed: latent spectral radius was 0.9021 and the measured future and transition residuals were finite.

Baseline: mean test MSE 0.0006831642931501847, std 0.00017069130263284838
Delta mean: -4.903324952465482e-05
Idea: mean test MSE 0.0006341310436255299, std 0.00015214794700785093
Idea wins: 6/8
P value: 0.36915

Benchmark result