# Hankel Residual Observer

- ID: 510
- Canonical URL: https://synthcore.org/idea/510/hankel-residual-observer
- API JSON: https://synthcore.org/api/idea/510.json
- API Markdown: https://synthcore.org/api/idea/510.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.07082](https://arxiv.org/abs/2607.07082)
- Category: architecture
- Solves: accuracy, stability, sample-efficiency
- ML areas: training, ssm, inference-speedup
- Math tags: linear-algebra, dynamical-systems, control-theory, optimization
- Ratings: usefulness 6/10; difficulty 4/10; novelty 7/10

## Idea description

Attach a model-free residual-dynamics observer to a neural multi-step forecaster. Instead of asking the network to relearn persistent periodic or autoregressive disturbances, maintain a Hankel dictionary of recent forecast errors and use ridge reconstruction to predict the next residual sequence online. Add the predicted residual to the network forecast with a confidence-dependent correction gain.

## Mathematical statement

For a vector sequence $x_t\in\mathbb{R}^{p}$, the order-$L$ Hankel matrix is $\mathcal{H}_{L}(x)=\begin{bmatrix}x_{1}&x_{2}&\dots&x_{T-L+1}\\x_{2}&x_{3}&\dots&x_{T-L+2}\\\vdots&\vdots&\ddots&\vdots\\x_{L}&x_{L+1}&\dots&x_{T}\end{bmatrix}$. MFDOOM constructs $D_p$ and $D_f$ from prediction-error samples, where $D_p$ contains past windows of length $T_{ini_d}$ and $D_f$ contains their matching future windows of length $H$. Given the latest error history $d_{ini}\in\mathbb{R}^{pT_{ini_d}}$, solve $g_d^*=\arg\min_g\|D_pg-d_{ini}\|_2^2+\lambda_g\|g\|_2^2$ and predict $\hat{\mathbf d}=D_fg_d^*$. The paper corrects predictions using $\mathbf y_{corr}=\mathbf y-\Gamma\mathbf d$, where $\mathbf y$ is the base prediction and $\Gamma$ is a correction-gain matrix. If residuals are stored as actual-minus-predicted, the equivalent implementation is $\hat y_{corr}=\hat y+\Gamma\hat d$. The reconstruction residual $\|D_pg_d^*-d_{ini}\|_2$ supplies a confidence test; disable or attenuate corrections when it exceeds a threshold.

## Key formulas

- $$\mathcal{H}_{L}(x):=\begin{bmatrix}x_{1}&x_{2}&\dots&x_{T-L+1}\\x_{2}&x_{3}&\dots&x_{T-L+2}\\\vdots&\vdots&\ddots&\vdots\\x_{L}&x_{L+1}&\dots&x_{T}\end{bmatrix}$$
- $$\begin{bmatrix}D_{p}\\D_{f}\end{bmatrix}\mathbf{g_{d}}=\begin{bmatrix}d_{ini}\\\mathbf{d}\end{bmatrix}$$
- $$g_d^*=\arg\min_g\left\|D_pg-d_{ini}\right\|_2^2+\lambda_g\left\|g\right\|_2^2,\qquad \hat{\mathbf d}=D_fg_d^*$$
- $$\hat y_{corr}=\hat y+\Gamma\hat d,\qquad \Gamma=\operatorname{diag}(\gamma_1,\ldots,\gamma_{pH})$$

## Implementation notes

Integrate the observer after a neural forecaster's multi-step output head and before inference-time prediction is returned or before the forecasting loss is evaluated. Let the network output $\hat y_{t+1:t+H}\in\mathbb{R}^{pH}$. When targets become available, compute the one-step residual $r_t=y_t-\hat y_t$ and append it to a bounded FIFO buffer. Once at least $T_{ini_d}+H$ residual vectors are available, construct dictionary columns from consecutive windows: $D_p[:,j]=[r_j;\ldots;r_{j+T_{ini_d}-1}]$ and $D_f[:,j]=[r_{j+T_{ini_d}};\ldots;r_{j+T_{ini_d}+H-1}]$. Retain the latest $K$ columns and rebuild or rank-update the Gram matrix. At prediction time, form $d_{ini}$ from the latest $T_{ini_d}$ residuals and solve $g=(D_p^TD_p+\lambda_gI)^{-1}D_p^Td_{ini}$ with Cholesky. Compute $\hat d=D_fg$. Define normalized mismatch $q=\|D_pg-d_{ini}\|_2/(\|d_{ini}\|_2+10^{-8})$ and set $\gamma_i=\operatorname{clip}(1-q/\tau,0,1)$, optionally multiplied by a learned horizon decay. Return $\hat y_{corr}=\hat y+\Gamma\hat d$. The Hankel construction and ridge solve come directly from the mathematical method; select $K$, $T_{ini_d}$, $\lambda_g$, $\epsilon$, and $\tau$ on a validation stream. First test a small GRU or TCN on synthetic forecasting with additive sinusoidal disturbances, then Electricity or Traffic. Compare the baseline at equal neural FLOPs and report multi-step RMSE, long-horizon error, observer CPU latency, and memory. The expected signal is lower error on persistent disturbances without degradation on clean data, with corrections automatically suppressed when dictionary mismatch is high.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: harms
- Verdict: Built a bounded Hankel residual observer with ridge reconstruction and confidence-scaled correction around a small GRU forecaster. Exact period-4 residuals were reconstructed with RMSE 8.6e-13, and disturbed-stream RMSE improved from 0.9907 to 0.8792. Clean-stream RMSE worsened from 0.05314 to 0.05495, so the benefit is promising but not robust across regimes.

### Mechanism check

- Verdict: Built a bounded Hankel residual observer with ridge reconstruction and confidence-scaled correction around a small GRU forecaster. Exact period-4 residuals were reconstructed with RMSE 8.6e-13, and disturbed-stream RMSE improved from 0.9907 to 0.8792. Clean-stream RMSE worsened from 0.05314 to 0.05495, so the benefit is promising but not robust across regimes.
- Confidence: 7/10
- Limitations: Only a synthetic AR(1) plus sinusoidal-disturbance stream was tested. No real benchmark, validation-based hyperparameter selection, multi-seed uncertainty, GPU timing, baseline latency, memory, or optimized incremental Gram updates were evaluated.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_hankel.py](https://synthcore.org/code/1363/bench_hankel.py)
- [bench_report.json](https://synthcore.org/code/1363/bench_report.json)
- [report.md](https://synthcore.org/code/1363/report.md)
- [report_bench_2026-09-04T172627.md](https://synthcore.org/code/1363/report_bench_2026-09-04T172627.md)
- [run_experiment.py](https://synthcore.org/code/1363/run_experiment.py)
- [Download all files as ZIP](https://synthcore.org/download/1363)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.