# Marginal-Stability Disorder Schedule

- ID: 1554
- Canonical URL: https://synthcore.org/idea/1554/marginal-stability-disorder-schedule
- API JSON: https://synthcore.org/api/idea/1554.json
- API Markdown: https://synthcore.org/api/idea/1554.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.28781](https://arxiv.org/abs/2607.28781)
- Category: training
- Solves: stability, accuracy, memory
- ML areas: rnn, optimizer, scheduler, training-dynamics
- Math tags: dynamical-systems, bifurcations, linear-algebra, statistical-mechanics
- Ratings: usefulness 7/10; difficulty 4/10; novelty 7/10

## Idea description

Use the disorder-controlled stability boundary as a training schedule. Start with strong damping so optimization is well behaved, then reduce the damping margin toward zero to create long-lived oscillatory state memory after the network has learned useful representations.

## Mathematical statement

Let r be the real part of the linearized eigenvalues and omega their imaginary part: r=sqrt(2/pi) J/sigma-1 and omega=sqrt(2/pi) K/sigma. The origin is stable when r<0 and unstable when r>0. In the stable linear regime, an oscillatory correlation has envelope exp(r t) and characteristic decay time tau=1/absolute value of r. Choose a positive damping margin epsilon and set sigma=sqrt(2/pi) J/(1-epsilon), which forces r=-epsilon. Annealing epsilon toward zero increases correlation time while retaining bounded dynamics. In discrete time, also enforce the Euler stability condition |1+Delta t(r plus or minus i omega)|<1.

## Key formulas

- $$r(\sigma)=\sqrt{\frac{2}{\pi}}\frac{J}{\sigma}-1,\qquad \omega(\sigma)=\sqrt{\frac{2}{\pi}}\frac{K}{\sigma},\qquad \tau(\sigma)=\frac{1}{|r(\sigma)|}$$
- $$\sigma_q=\frac{\sqrt{2/\pi}\,J_q}{1-\epsilon_q},\qquad r_q=-\epsilon_q,\qquad 0<\epsilon_q<1$$
- $$C(\ell)\approx C(0)\exp(r\ell\Delta t)\cos(\omega\ell\Delta t)$$

## Implementation notes

Apply the schedule to sigma in the oscillatory recurrent cell, or equivalently divide the recurrent preactivation by a time-dependent gain. At optimizer step q, estimate the effective self-coupling J_q from the spectral norm or average diagonal gain of the self-coupling block, then set sigma_q=sqrt(2/pi) J_q divided by (1-epsilon_q), using stop-gradient through this schedule. Start with epsilon_init=0.3 and anneal exponentially or by cosine to epsilon_final=0.02 over T training steps. Pseudocode is: J_gain=estimate_gain(W_self); epsilon=epsilon_final+(epsilon_init-epsilon_final)*exp(-q/T); sigma=sqrt(2/pi)*J_gain/(1-epsilon); h=oscillatory_cell(h,u,J_gain,K,sigma). Clip sigma to a positive range and separately check the discrete Euler modulus because large K or Delta t can destabilize an otherwise stable continuous-time system. The paper-derived quantities are the eigenvalue formulas and the disorder-to-gain relation; empirical quantities are the effective Jacobian, actual autocorrelation, and gradient variance. First test on the copy-memory and adding tasks with a small RNN, comparing fixed sigma, spectral normalization, and the proposed schedule. The quantitative prediction is that the measured correlation decay time should track tau=1/epsilon during annealing, while the autocorrelation envelope follows exp(-epsilon times lag times Delta t). Crossing epsilon below zero or violating the Euler modulus should produce a sharp increase in hidden-state norm and gradient variance. Require measured decay rates to agree with the predicted value within 20 percent.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: inconclusive
- Verdict: Built a reproducible NumPy MVP implementing the disorder-controlled sigma schedule and a matched recurrent memory toy task. The mechanism was confirmed: measured damping matched r=-epsilon, correlation-time error stayed below 1.3% across epsilon=0.30 to 0.02, and the analytic Euler boundary matched numerical spectral-radius search to machine precision; negative epsilon produced a positive eigenvalue and an oversized Euler step produced spectral radius 1.554. The schedule did not produce a task-performance win in this tiny run (accuracy 0.690 vs 0.700 for fixed sigma), so the evidence supports the stability/memory mechanism but not improved learning.

### Mechanism check

- Verdict: Built a reproducible NumPy MVP implementing the disorder-controlled sigma schedule and a matched recurrent memory toy task. The mechanism was confirmed: measured damping matched r=-epsilon, correlation-time error stayed below 1.3% across epsilon=0.30 to 0.02, and the analytic Euler boundary matched numerical spectral-radius search to machine precision; negative epsilon produced a positive eigenvalue and an oversized Euler step produced spectral radius 1.554. The schedule did not produce a task-performance win in this tiny run (accuracy 0.690 vs 0.700 for fixed sigma), so the evidence supports the stability/memory mechanism but not improved learning.
- Confidence: 9/10
- Limitations: Only a small NumPy memory task was tested; copy-memory and adding benchmarks, spectral-normalization baseline, multiple seeds, gradient-variance analysis, and larger learned RNNs were not tested. The oscillator verification uses the specified linearized model directly rather than estimating an empirical Jacobian of a nonlinear recurrent network.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.64285
- Paired wins: 6/8
- Benchmark verdict: no significant win

## Artifacts

- [bench_report.json](https://synthcore.org/code/473/bench_report.json)
- [bench_run.py](https://synthcore.org/code/473/bench_run.py)
- [experiment.py](https://synthcore.org/code/473/experiment.py)
- [report.md](https://synthcore.org/code/473/report.md)
- [report_bench_2026-08-31T142607.md](https://synthcore.org/code/473/report_bench_2026-08-31T142607.md)
- [results.json](https://synthcore.org/code/473/results.json)
- [stage2_bench.py](https://synthcore.org/code/473/stage2_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/473)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.