# Greedy Singular-Value Delay Scheduler

- ID: 2273
- Canonical URL: https://synthcore.org/idea/2273/greedy-singular-value-delay-scheduler
- API JSON: https://synthcore.org/api/idea/2273.json
- API Markdown: https://synthcore.org/api/idea/2273.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2608.14001](https://arxiv.org/abs/2608.14001)
- Category: training
- Solves: stability, memory, accuracy
- ML areas: ssm, rnn, world-model, memory, regularization
- Math tags: dynamical-systems, linear-algebra, spectral-theory, control-theory
- Ratings: usefulness 7/10; difficulty 5/10; novelty 8/10

## Idea description

Use the observability margin to choose which delay taps to retain under a fixed memory or computation budget. Add a candidate delay only when it substantially increases the smallest singular value of the delay map, converting the paper's large-delay asymptotic result into an adaptive receptive-field construction for sequence models.

## Mathematical statement

Let \(T_m=\{t_1,\ldots,t_m\}\) be the current delay set and let \(O_m\) be its delay-observation matrix. Adding candidate delay \(t\) appends row \(r(t)=c^{\mathsf{T}}e^{-At}\), producing \(O_{m+1}(t)=\begin{bmatrix}O_m\\r(t)\end{bmatrix}\). The stable reconstruction margin is \(\sigma_{\min}(O_m)\), because \(\|O_m^{\dagger}\|_2=1/\sigma_{\min}(O_m)\). Select the delay with the largest increase in this margin. For a nonlinear neural encoder, use the local delay Jacobian \(J_T(z)=\partial\Psi_T(z)/\partial z\), where \(z\) is the latent state, and maximize its batch-averaged smallest singular value.

## Key formulas

- $$O_{m+1}(t)=\begin{bmatrix}O_m\\r(t)\end{bmatrix},\qquad r(t)=c^{\mathsf{T}}e^{-At},\qquad \|O_{m+1}^{\dagger}\|_2=\frac{1}{\sigma_{\min}(O_{m+1})}.$$
- $$t_{m+1}=\arg\max_{t\in\mathcal{C}\setminus T_m}\left[\sigma_{\min}(O_{m+1}(t))-\sigma_{\min}(O_m)\right].$$
- $$\text{stop if }\max_{t\in\mathcal{C}\setminus T_m}\left[\sigma_{\min}(O_{m+1}(t))-\sigma_{\min}(O_m)\right]<\delta.$$
- $$\mathcal{L}_{\mathrm{delay}}=-\frac{1}{B}\sum_{b=1}^{B}\log\left(\sigma_{\min}(J_T(z_b))^2+\epsilon\right)+\gamma m.$$

## Implementation notes

Integration point: apply the scheduler to the receptive field of a delay MLP, neural CDE, recurrent encoder, or latent SSM. Construct a candidate delay grid with short linear spacing for fast dynamics and logarithmic spacing for slow dynamics. At initialization, begin with the current observation and greedily add candidate taps. Under a fixed budget, retain only the selected delays and use them throughout training.

Pseudocode: initialize \(T_1=\{0\}\); compute \(O_1\); for every candidate \(t\), append \(c^{\mathsf{T}}\exp(-At)\), calculate \(\sigma_{\min}\), and select the candidate with the largest gain; repeat until the budget is reached or the gain falls below \(\delta\). In a nonlinear model, calculate the encoder Jacobian with automatic differentiation on a minibatch and use its smallest singular value. Recompute selection periodically if the dynamics model changes. The known-system matrix and singular-value calculation are exact; learned matrices, Jacobians, threshold \(\delta\), and penalty \(\gamma\) are empirical.

First experiment: train a delay MLP on a damped oscillator and a two-frequency signal with randomly missing observations. Compare 4, 8, and 16 uniformly spaced taps against greedy singular-value-selected taps with equal tap counts. Measure reconstruction, 100-step rollout, memory, and the singular-value spectrum. The predicted signature is an elbow: error should decrease rapidly while \(\sigma_{\min}\) rises, then show diminishing returns once the marginal gain is below \(\delta\). Selected delays should cluster near inverse modal time scales. At matched rollout error, the scheduler should reduce retained taps by at least 25 percent; absence of a spectral elbow or correlation between \(1/\sigma_{\min}^2\) and error falsifies the proposed mechanism.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: no_effect
- Verdict: Built an exact matrix-exponential greedy delay scheduler with fixed-budget selection and stopping-threshold support. The mechanism manifested across sweeps: row-appending margins were monotone, selected delays tracked pi/(2 omega) with maximum error 0.0385, and reconstruction MSE scaled with noise variance; MSE versus 1/sigma_min^2 had correlation 0.986. Greedy selection improved linear toy MSE by 21.7% at 4 taps and 9.7% at 8 taps versus uniform delays, but no neural delay-MLP or rollout win was tested.

### Mechanism check

- Verdict: Built an exact matrix-exponential greedy delay scheduler with fixed-budget selection and stopping-threshold support. The mechanism manifested across sweeps: row-appending margins were monotone, selected delays tracked pi/(2 omega) with maximum error 0.0385, and reconstruction MSE scaled with noise variance; MSE versus 1/sigma_min^2 had correlation 0.986. Greedy selection improved linear toy MSE by 21.7% at 4 taps and 9.7% at 8 taps versus uniform delays, but no neural delay-MLP or rollout win was tested.
- Confidence: 8/10
- Limitations: Only the exact two-dimensional damped oscillator was tested. Nonlinear delay MLPs, missing observations, two-frequency signals, 100-step rollouts, learned/local Jacobian scheduling, profiling, and broader systems were not tested. The candidate grid also permits near-duplicate delays.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.43475
- Paired wins: 3/8
- Benchmark verdict: no significant win

## Artifacts

- [bench_report.json](https://synthcore.org/code/906/bench_report.json)
- [experiment.py](https://synthcore.org/code/906/experiment.py)
- [report.md](https://synthcore.org/code/906/report.md)
- [report_bench_2026-09-01T143702.md](https://synthcore.org/code/906/report_bench_2026-09-01T143702.md)
- [results.json](https://synthcore.org/code/906/results.json)
- [stage2_bench.py](https://synthcore.org/code/906/stage2_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/906)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.