# Conditioned Irregular-Delay State Encoder

- ID: 2272
- Canonical URL: https://synthcore.org/idea/2272/conditioned-irregular-delay-state-encoder
- API JSON: https://synthcore.org/api/idea/2272.json
- API Markdown: https://synthcore.org/api/idea/2272.md
- Verification status: failed_benchmark
- Source: [arXiv:2608.14001](https://arxiv.org/abs/2608.14001)
- Category: architecture
- Solves: stability, accuracy, sample-efficiency
- ML areas: ssm, rnn, world-model, regularization
- Math tags: dynamical-systems, linear-algebra, spectral-theory, control-theory
- Ratings: usefulness 8/10; difficulty 6/10; novelty 7/10

## Idea description

Replace uniformly spaced history taps in a neural state-space encoder with a fixed or learned set of non-uniform delays. Regularize the resulting delay-observation matrix to have a large smallest singular value, which makes latent-state reconstruction less sensitive to irregular timestamps and observation noise. This is directly applicable to event-based data, missing timestamps, and systems with multiple time scales.

## Mathematical statement

Consider continuous-time linear dynamics \(\dot{x}=Ax\), where \(x\in\mathbb{R}^{n}\), and scalar observation \(y(t)=c^{\mathsf{T}}x(t)\), where \(A\in\mathbb{R}^{n\times n}\) and \(c\in\mathbb{R}^{n}\). For delays \(0\leq t_1<\cdots<t_\ell\), the delay vector satisfies \(y(t-t_i)=c^{\mathsf{T}}e^{-At_i}x(t)\). Therefore the delay map is \(\Psi_T(x)=O_Tx\), where row \(i\) of \(O_T\in\mathbb{R}^{\ell\times n}\) is \(c^{\mathsf{T}}e^{-At_i}\). Full column rank gives local linear observability. Quantitatively, the smallest singular value \(\sigma_{\min}(O_T)\) controls noise amplification: for measurement noise \(\varepsilon\), least-squares reconstruction satisfies \(\|\hat{x}-x\|_2\leq\|\varepsilon\|_2/\sigma_{\min}(O_T)\). The condition number \(\kappa(O_T)=\sigma_{\max}(O_T)/\sigma_{\min}(O_T)\) measures sensitivity to perturbations.

## Key formulas

- $$\Psi_T(x(t))=\begin{bmatrix}c^{\mathsf{T}}e^{-At_1}\\ \vdots \\ c^{\mathsf{T}}e^{-At_\ell}\end{bmatrix}x(t)=O_Tx(t),\qquad O_T(i,:)=c^{\mathsf{T}}e^{-At_i}.$$
- $$\hat{x}=O_T^{\dagger}y_T,\qquad y_T=O_Tx+\varepsilon,\qquad \|\hat{x}-x\|_2\leq\frac{\|\varepsilon\|_2}{\sigma_{\min}(O_T)}.$$
- $$\mathcal{L}_{\mathrm{obs}}=\frac{1}{\sigma_{\min}(O_T)^2+\epsilon}+\alpha\frac{\sigma_{\max}(O_T)^2}{\sigma_{\min}(O_T)^2+\epsilon}.$$
- $$A=V\Lambda V^{-1}\quad\Longrightarrow\quad O_TV=\left[c^{\mathsf{T}}V\right]_j e^{-\lambda_jt_i},$$

## Implementation notes

Integration point: build an encoder for timestamped observations \((y_k,s_k)\) in a neural state-space model. At time \(s_k\), select the previous \(\ell\) observations and compute delays \(t_i=s_k-s_{k-i}\). Feed the resulting irregular delay vector to an MLP or linear projection that initializes the latent state of an SSM. If a continuous-time latent model has learned matrix \(A\) and observation vector \(c\), compute \(O_T\) with matrix exponentials and add \(\beta\mathcal{L}_{\mathrm{obs}}\) to the prediction loss. For a nonlinear model, replace \(O_T\) by the Jacobian of the delay encoder with respect to the latent state, estimated by automatic differentiation.

Pseudocode: sample a batch of timestamped histories; compute all delays; evaluate \(E_i=\exp(-At_i)\); stack \(c^{\mathsf{T}}E_i\) into \(O_T\); calculate its singular values; add the observability penalty to the task loss; update model parameters. Optionally make delays trainable through positive increments \(t_i=\sum_{j\leq i}\operatorname{softplus}(u_j)\). Matrix exponentials and singular values are computed from the paper's construction; \(A\), \(c\), and nonlinear Jacobians are estimated or learned in the neural implementation.

First experiment: use a two-dimensional damped oscillator and Lorenz-63 with Poisson-distributed sampling intervals. Compare a GRU, a uniformly sampled delay MLP, and this irregular-delay encoder on reconstruction, one-step prediction, and 50-step rollout under additive observation noise. Sweep noise level and number of delays. The quantitative prediction is that reconstruction error scales approximately with \(1/\sigma_{\min}(O_T)^2\), and rollout instability should appear when \(\sigma_{\min}(O_T)\) approaches the effective noise floor. The measured error versus inverse singular value should have positive Spearman correlation above 0.7, and the empirical instability boundary should agree with the singular-value prediction within 20 percent.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: no_effect
- Verdict: Implemented exact continuous-time delay observability and timestamp-conditioned least-squares reconstruction in run_experiment.py. The mechanism manifested: noise scaling had observed log-log slope 0.9992 versus predicted 1.0, reconstruction error correlated with the predicted inverse-Gram noise factor at Spearman rho=0.943, the pseudoinverse bound held, and optimized delay placement increased sigma_min by 43.4%. In the Poisson-irregular toy comparison, using actual timestamps reduced latent reconstruction error by 94.1–97.3% versus a fixed uniform-delay decoder, though this comparison gives the proposed method the correct timestamps and is not a learned neural-model win.

### Mechanism check

- Verdict: Implemented exact continuous-time delay observability and timestamp-conditioned least-squares reconstruction in run_experiment.py. The mechanism manifested: noise scaling had observed log-log slope 0.9992 versus predicted 1.0, reconstruction error correlated with the predicted inverse-Gram noise factor at Spearman rho=0.943, the pseudoinverse bound held, and optimized delay placement increased sigma_min by 43.4%. In the Poisson-irregular toy comparison, using actual timestamps reduced latent reconstruction error by 94.1–97.3% versus a fixed uniform-delay decoder, though this comparison gives the proposed method the correct timestamps and is not a learned neural-model win.
- Confidence: 9/10
- Limitations: Only a 2D linear damped oscillator was tested. No GRU, MLP, Lorenz-63, learned delays, nonlinear Jacobian regularization, prediction loss, rollout stability, runtime/FLOP, or missing-data neural training experiment was evaluated; the baseline comparison is an oracle-style linear timestamp mismatch test.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.1247937603119844
- Paired wins: 5/8
- Benchmark verdict: idea better (not significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/798/bench_report.json)
- [report.md](https://synthcore.org/code/798/report.md)
- [report_bench_2026-09-01T105839.md](https://synthcore.org/code/798/report_bench_2026-09-01T105839.md)
- [results.json](https://synthcore.org/code/798/results.json)
- [run_experiment.py](https://synthcore.org/code/798/run_experiment.py)
- [stage2_bench.py](https://synthcore.org/code/798/stage2_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/798)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.