✗ Failed on benchmark 2026

Conditioned Irregular-Delay State Encoder

Implementation & benchmark of arXiv:2608.14001 — Stable Takens' Embedding Theorem for Non-Uniformly-Sampled Linear Systems

Usefulness8/10

Difficulty6/10

Novelty7/10

Architecture Stability Accuracy Sample efficiency Ssm Rnn World model Regularization Dynamical systems Linear algebra Spectral theory Control theory

Source paper: Stable Takens' Embedding Theorem for Non-Uniformly-Sampled Linear Systems arXiv:2608.14001 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace uniformly spaced history taps in a neural state-space encoder with a fixed or learned set of non-uniform delays. Regularize the resulting delay-observation matrix to have a large smallest singular value, which makes latent-state reconstruction less sensitive to irregular timestamps and observation noise. This is directly applicable to event-based data, missing timestamps, and systems with multiple time scales.

Formulas

$$\Psi_T(x(t))=\begin{bmatrix}c^{\mathsf{T}}e^{-At_1}\\ \vdots \\ c^{\mathsf{T}}e^{-At_\ell}\end{bmatrix}x(t)=O_Tx(t),\qquad O_T(i,:)=c^{\mathsf{T}}e^{-At_i}.$$

$$\hat{x}=O_T^{\dagger}y_T,\qquad y_T=O_Tx+\varepsilon,\qquad \|\hat{x}-x\|_2\leq\frac{\|\varepsilon\|_2}{\sigma_{\min}(O_T)}.$$

$$\mathcal{L}_{\mathrm{obs}}=\frac{1}{\sigma_{\min}(O_T)^2+\epsilon}+\alpha\frac{\sigma_{\max}(O_T)^2}{\sigma_{\min}(O_T)^2+\epsilon}.$$

$$A=V\Lambda V^{-1}\quad\Longrightarrow\quad O_TV=\left[c^{\mathsf{T}}V\right]_j e^{-\lambda_jt_i},$$

Mathematical statement

Consider continuous-time linear dynamics $\dot{x}=Ax$, where $x\in\mathbb{R}^{n}$, and scalar observation $y(t)=c^{\mathsf{T}}x(t)$, where $A\in\mathbb{R}^{n\times n}$ and $c\in\mathbb{R}^{n}$. For delays $0\leq t_1<\cdots<t_\ell$, the delay vector satisfies $y(t-t_i)=c^{\mathsf{T}}e^{-At_i}x(t)$. Therefore the delay map is $\Psi_T(x)=O_Tx$, where row $i$ of $O_T\in\mathbb{R}^{\ell\times n}$ is $c^{\mathsf{T}}e^{-At_i}$. Full column rank gives local linear observability. Quantitatively, the smallest singular value $\sigma_{\min}(O_T)$ controls noise amplification: for measurement noise $\varepsilon$, least-squares reconstruction satisfies $\|\hat{x}-x\|_2\leq\|\varepsilon\|_2/\sigma_{\min}(O_T)$. The condition number $\kappa(O_T)=\sigma_{\max}(O_T)/\sigma_{\min}(O_T)$ measures sensitivity to perturbations.

Implementation notes

Integration point: build an encoder for timestamped observations $(y_k,s_k)$ in a neural state-space model. At time $s_k$, select the previous $\ell$ observations and compute delays $t_i=s_k-s_{k-i}$. Feed the resulting irregular delay vector to an MLP or linear projection that initializes the latent state of an SSM. If a continuous-time latent model has learned matrix $A$ and observation vector $c$, compute $O_T$ with matrix exponentials and add $\beta\mathcal{L}_{\mathrm{obs}}$ to the prediction loss. For a nonlinear model, replace $O_T$ by the Jacobian of the delay encoder with respect to the latent state, estimated by automatic differentiation.

Pseudocode: sample a batch of timestamped histories; compute all delays; evaluate $E_i=\exp(-At_i)$; stack $c^{\mathsf{T}}E_i$ into $O_T$; calculate its singular values; add the observability penalty to the task loss; update model parameters. Optionally make delays trainable through positive increments $t_i=\sum_{j\leq i}\operatorname{softplus}(u_j)$. Matrix exponentials and singular values are computed from the paper's construction; $A$, $c$, and nonlinear Jacobians are estimated or learned in the neural implementation.

First experiment: use a two-dimensional damped oscillator and Lorenz-63 with Poisson-distributed sampling intervals. Compare a GRU, a uniformly sampled delay MLP, and this irregular-delay encoder on reconstruction, one-step prediction, and 50-step rollout under additive observation noise. Sweep noise level and number of delays. The quantitative prediction is that reconstruction error scales approximately with $1/\sigma_{\min}(O_T)^2$, and rollout instability should appear when $\sigma_{\min}(O_T)$ approaches the effective noise floor. The measured error versus inverse singular value should have positive Spearman correlation above 0.7, and the empirical instability boundary should agree with the singular-value prediction within 20 percent.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: No measurable effect

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Implemented exact continuous-time delay observability and timestamp-conditioned least-squares reconstruction in run_experiment.py. The mechanism manifested: noise scaling had observed log-log slope 0.9992 versus predicted 1.0, reconstruction error correlated with the predicted inverse-Gram noise factor at Spearman rho=0.943, the pseudoinverse bound held, and optimized delay placement increased sigma_min by 43.4%. In the Poisson-irregular toy comparison, using actual timestamps reduced latent reconstruction error by 94.1–97.3% versus a fixed uniform-delay decoder, though this comparison gives the proposed method the correct timestamps and is not a learned neural-model win.

Agent confidence: 9/10

Baseline: Fixed nominal uniform-delay least-squares decoder; mean latent error for mean timestamp gaps [0.08, 0.15, 0.30, 0.50] was [1.0613, 0.7987, 0.6477, 0.9155].
Idea: Timestamp-conditioned least-squares decoder; corresponding errors were [0.0630, 0.0359, 0.0289, 0.0244], relative improvements [94.1%, 95.5%, 95.5%, 97.3%]. Core checks: noise slope 0.9992, Spearman rho 0.9429, maximum normalized bound ratio 0.9927, delay-design sigma_min improvement 1.434x.

Limitations: Only a 2D linear damped oscillator was tested. No GRU, MLP, Lorenz-63, learned delays, nonlinear Jacobian regularization, prediction loss, rollout stability, runtime/FLOP, or missing-data neural training experiment was evaluated; the baseline comparison is an oracle-style linear timestamp mismatch test.

How to run: python3 run_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.14001) Sep 2, 2026

Ahead of the baseline (-0.0013 metric points) but not statistically significant (p=0.1248, wins 5 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: shared DelayGRU
Paired seeds: 8
Baseline mean: 0
Idea mean: 0.1711
Effect (Δ): -0.0013 (n/a; negative = idea better)
Wins: 5 / 8 paired seeds
p-value: 0.1248 (permutation test, 20 000 shuffles)
Smallest detectable effect: n/a
Mechanism: Not confirmed ✗
Practical effect: No effect
Baseline tuning: swept over 0 configs

Limitations:

This is not official harness evidence because /home/maxwelhelp/all/math2nn/bench and README.md were unavailable. The local benchmark used a synthetic damped oscillator rather than the official actuated-pendulum dataset, and no official make_report or sweep_baseline call was possible.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Failed ✗

Agent confidence: 10/10

Built a local dynamics benchmark with matched GRU systems: the baseline uses fixed uniform-delay features, while the idea uses observed irregular delays. Across 8 paired seeds, the idea slightly improved test MSE (0.17109 vs 0.17238; delta -0.00129), but the permutation p-value was 0.1248, so the improvement was not significant. The mechanism signature was not confirmed because observed noise-scaling slopes were approximately 0.007 rather than the predicted 1.0; additionally, the official benchmark package and README were absent from the advertised path.

Baseline: 8-seed test MSE mean 0.1723834295, std 0.0237546733; best lr=0.006, wd=0.0.
Idea: 8-seed test MSE mean 0.1710888054; 5/8 paired wins; delta=-0.0012946241; permutation p=0.1247937603; not significant.
Mechanism signature: Observed log-log noise slopes: baseline 0.0070741, idea 0.0070607; confirmed=false.

Benchmark result