△ Mechanism confirmed, baseline not beaten 2026

Greedy Singular-Value Delay Scheduler

Implementation & benchmark of arXiv:2608.14001 — Stable Takens' Embedding Theorem for Non-Uniformly-Sampled Linear Systems

Usefulness7/10

Difficulty5/10

Novelty8/10

Training Stability Memory Accuracy Ssm Rnn World model Memory Regularization Dynamical systems Linear algebra Spectral theory Control theory

Source paper: Stable Takens' Embedding Theorem for Non-Uniformly-Sampled Linear Systems arXiv:2608.14001 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Use the observability margin to choose which delay taps to retain under a fixed memory or computation budget. Add a candidate delay only when it substantially increases the smallest singular value of the delay map, converting the paper's large-delay asymptotic result into an adaptive receptive-field construction for sequence models.

Formulas

$$O_{m+1}(t)=\begin{bmatrix}O_m\\r(t)\end{bmatrix},\qquad r(t)=c^{\mathsf{T}}e^{-At},\qquad \|O_{m+1}^{\dagger}\|_2=\frac{1}{\sigma_{\min}(O_{m+1})}.$$

$$t_{m+1}=\arg\max_{t\in\mathcal{C}\setminus T_m}\left[\sigma_{\min}(O_{m+1}(t))-\sigma_{\min}(O_m)\right].$$

$$\text{stop if }\max_{t\in\mathcal{C}\setminus T_m}\left[\sigma_{\min}(O_{m+1}(t))-\sigma_{\min}(O_m)\right]<\delta.$$

$$\mathcal{L}_{\mathrm{delay}}=-\frac{1}{B}\sum_{b=1}^{B}\log\left(\sigma_{\min}(J_T(z_b))^2+\epsilon\right)+\gamma m.$$

Mathematical statement

Let $T_m=\{t_1,\ldots,t_m\}$ be the current delay set and let $O_m$ be its delay-observation matrix. Adding candidate delay $t$ appends row $r(t)=c^{\mathsf{T}}e^{-At}$, producing $O_{m+1}(t)=\begin{bmatrix}O_m\\r(t)\end{bmatrix}$. The stable reconstruction margin is $\sigma_{\min}(O_m)$, because $\|O_m^{\dagger}\|_2=1/\sigma_{\min}(O_m)$. Select the delay with the largest increase in this margin. For a nonlinear neural encoder, use the local delay Jacobian $J_T(z)=\partial\Psi_T(z)/\partial z$, where $z$ is the latent state, and maximize its batch-averaged smallest singular value.

Implementation notes

Integration point: apply the scheduler to the receptive field of a delay MLP, neural CDE, recurrent encoder, or latent SSM. Construct a candidate delay grid with short linear spacing for fast dynamics and logarithmic spacing for slow dynamics. At initialization, begin with the current observation and greedily add candidate taps. Under a fixed budget, retain only the selected delays and use them throughout training.

Pseudocode: initialize $T_1=\{0\}$; compute $O_1$; for every candidate $t$, append $c^{\mathsf{T}}\exp(-At)$, calculate $\sigma_{\min}$, and select the candidate with the largest gain; repeat until the budget is reached or the gain falls below $\delta$. In a nonlinear model, calculate the encoder Jacobian with automatic differentiation on a minibatch and use its smallest singular value. Recompute selection periodically if the dynamics model changes. The known-system matrix and singular-value calculation are exact; learned matrices, Jacobians, threshold $\delta$, and penalty $\gamma$ are empirical.

First experiment: train a delay MLP on a damped oscillator and a two-frequency signal with randomly missing observations. Compare 4, 8, and 16 uniformly spaced taps against greedy singular-value-selected taps with equal tap counts. Measure reconstruction, 100-step rollout, memory, and the singular-value spectrum. The predicted signature is an elbow: error should decrease rapidly while $\sigma_{\min}$ rises, then show diminishing returns once the marginal gain is below $\delta$. Selected delays should cluster near inverse modal time scales. At matched rollout error, the scheduler should reduce retained taps by at least 25 percent; absence of a spectral elbow or correlation between $1/\sigma_{\min}^2$ and error falsifies the proposed mechanism.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: No measurable effect

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built an exact matrix-exponential greedy delay scheduler with fixed-budget selection and stopping-threshold support. The mechanism manifested across sweeps: row-appending margins were monotone, selected delays tracked pi/(2 omega) with maximum error 0.0385, and reconstruction MSE scaled with noise variance; MSE versus 1/sigma_min^2 had correlation 0.986. Greedy selection improved linear toy MSE by 21.7% at 4 taps and 9.7% at 8 taps versus uniform delays, but no neural delay-MLP or rollout win was tested.

Agent confidence: 8/10

Baseline: Uniform delays: at 4 taps sigma_min=1.1093 and MSE=0.00061785; at 8 taps sigma_min=1.5816 and MSE=0.00030887; at 16 taps sigma_min=2.2761 and MSE=0.00015513.
Idea: Greedy delays: at 4 taps sigma_min=1.3124 and MSE=0.00048366; at 8 taps sigma_min=1.7912 and MSE=0.00027900; at 16 taps sigma_min=2.4233 and MSE=0.00015152. Quarter-period sweep maximum absolute error=0.0385; noise-scaling ratio CV=2.1e-16; MSE versus inverse-margin-squared correlation=0.9858.

Limitations: Only the exact two-dimensional damped oscillator was tested. Nonlinear delay MLPs, missing observations, two-frequency signals, 100-step rollouts, learned/local Jacobian scheduling, profiling, and broader systems were not tested. The candidate grid also permits near-duplicate delays.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.14001) Sep 2, 2026

No significant difference vs. the tuned baseline (0 metric points, p=0.4348, wins 3 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0
Idea mean: 0
Effect (Δ): 0 (n/a; negative = idea better)
Wins: 3 / 8 paired seeds
p-value: 0.4348 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±10.5%
Mechanism: Confirmed ✓
Practical effect: No effect
Baseline tuning: swept over 0 configs

Limitations:

Only the built-in dynamics track was tested. No sequence track, custom delay-MLP, periodic reselection, missing-observation augmentation, rollout-horizon metric, or larger tap-budget experiment was run. The nominal margin uses a fixed linearized oscillator rather than learned system identification.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Failed ✗

Agent confidence: 9/10

Implemented and ran the registered built-in dynamics track with matched rnn_small systems: uniform four-tap baseline versus greedy singular-value-selected four-tap receptive field. Across 8 paired seeds, the idea had mean test MSE 0.0013303106 versus baseline 0.0013361162, but the paired permutation test was not significant (p=0.43475), so worked=false.

Baseline: Best lr=0.006; 8-seed mean test MSE=0.001336116241873242
Idea: Best lr=0.006; 8-seed mean test MSE=0.0013303105588420294
Paired delta: 4.8438116209581494e-05 (idea minus baseline)
Permutation p: 0.43475

Benchmark result