# Phase-Delay Spectral Margin for Attractor RNNs

- ID: 3035
- Canonical URL: https://synthcore.org/idea/3035/phase-delay-spectral-margin-for-attractor-rnns
- API JSON: https://synthcore.org/api/idea/3035.json
- API Markdown: https://synthcore.org/api/idea/3035.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2609.02047](https://arxiv.org/abs/2609.02047)
- Category: dynamics
- Solves: stability, generalization
- ML areas: rnn, ssm, optimizer, training-dynamics
- Math tags: dynamical-systems, spectral-theory, linear-algebra, control-theory
- Ratings: usefulness 8/10; difficulty 5/10; novelty 7/10

## Idea description

Build a continuous-time or discretized recurrent network whose interaction graph has trainable magnitudes and phase delays, then regularize the spectrum of the phase-corrected interaction matrix around each desired latent phase-locked state. The cosine-weighted composite matrix determines whether perturbations contract or grow, providing a computable stability margin instead of relying only on empirical exploding-gradient detection.

## Mathematical statement

For a phase network with node phases \(\theta_i\), adjacency weights \(A_{ij}\ge 0\), coupling strength \(K>0\), natural frequencies \(\omega_i\), and edge delay \(\alpha_{ij}\), use \(\dot\theta_i=\omega_i+K\sum_j A_{ij}\sin(\theta_j-\theta_i-\alpha_{ij})\). A phase-locked state \(\theta_i^*(t)=\Omega t+\psi_i\) has perturbation Jacobian \(J=-K L_C\), where \(C_{ij}=A_{ij}\cos(\psi_j-\psi_i-\alpha_{ij})\) and \(L_C=\operatorname{diag}(C\mathbf 1)-C\). The composite spectrum determines local stability: every non-gauge eigenvalue of \(J\) must have negative real part, equivalently \(\operatorname{Re}\lambda_k(L_C)>0\). For a discrete Euler update with step size \(\eta\), the local map is \(\delta\theta_{t+1}=(I+\eta J)\delta\theta_t\), requiring \(|1-\eta K\lambda_k(L_C)|<1\) for every mode.

## Key formulas

- $$\dot\theta_i=\omega_i+K\sum_j A_{ij}\sin(\theta_j-\theta_i-\alpha_{ij})$$
- $$C_{ij}=A_{ij}\cos(\psi_j-\psi_i-\alpha_{ij}),\qquad L_C=\operatorname{diag}(C\mathbf 1)-C,\qquad J=-K L_C$$
- $$\operatorname{Re}\lambda_k(L_C)>0\;(k\ne\mathrm{gauge}),\qquad |1-\eta K\lambda_k(L_C)|<1$$
- $$\mathcal L_{\mathrm{spec}}=\operatorname{softplus}\!\left(\gamma-\min_{k\ne\mathrm{gauge}}\operatorname{Re}\lambda_k(L_C)\right)+\beta\max_k\left(|1-\eta K\lambda_k(L_C)|-1\right)_+^2$$

## Implementation notes

1. Integration point: use a phase-valued recurrent cell or neural ODE with hidden state \(\theta\in\mathbb R^n\). Parameterize nonnegative edge magnitudes by \(A_{ij}=\operatorname{softplus}(a_{ij})\), and learn delays \(\alpha_{ij}\) modulo \(2\pi\). Add an input-dependent frequency term \(\omega_i=W_i x_t+b_i\). 2. Pseudocode: estimate a candidate locked state \(\psi\) by averaging the latent phase trajectory after teacher forcing; compute \(C_{ij}=A_{ij}\cos(\psi_j-\psi_i-\alpha_{ij})\); form \(L_C=\operatorname{diag}(C\mathbf 1)-C\); estimate eigenvalues using a dense eigensolver for small models or Arnoldi/power iterations for sparse models; add \(\mathcal L_{\mathrm{spec}}\) to the task loss; backpropagate through the eigensolver or use a detached spectral estimate for a slower control update. At inference, choose \(\eta\) below the largest value satisfying \(\max_k|1-\eta K\lambda_k|<1\). 3. Computed from the paper's mechanism: the cosine phase-delay composite matrix and its spectral stability test. Estimated empirically: the candidate locked state and basin volume. 4. First cheap experiment: train a 16-unit phase RNN on adding-machine sequences and compare a standard recurrent cell with the spectral-regularized cell at matched parameter count. Sweep \(\eta K\), record hidden-state norms, gradient norms, and recovery after perturbing a locked trajectory. The quantitative prediction is a sharp instability boundary at \(|1-\eta K\lambda_k|=1\); for an approximately real spectrum, measured divergence should begin within 20% of \(\eta_c=2/(K\lambda_{\max}(L_C))\). Increasing the minimum real eigenvalue by \(\gamma\) should increase the perturbation-decay rate approximately as \(K\min_{k\ne\mathrm{gauge}}\operatorname{Re}\lambda_k(L_C)\).

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: inconclusive
- Verdict: Built a phase-delay RNN with trainable nonnegative edge magnitudes, bounded phase delays, differentiable composite-Laplacian spectral regularization, and Euler stability diagnostics. The toy check reproduced the predicted boundary: amplification was 0.60 at 0.8ηc, 1.00 at ηc, and 1.40 at 1.2ηc. In the matched mini-run, the idea improved the minimum real spectral margin from 0.341 to 3.295, reduced maximum Euler amplification from 0.959 to 0.606, and reduced 12-step perturbation gain from 0.334 to 0.120, but task MSE worsened from 0.000172 to 0.00561; thus the stability effect is real, but task-performance benefit was not demonstrated.

### Mechanism check

- Verdict: Built a phase-delay RNN with trainable nonnegative edge magnitudes, bounded phase delays, differentiable composite-Laplacian spectral regularization, and Euler stability diagnostics. The toy check reproduced the predicted boundary: amplification was 0.60 at 0.8ηc, 1.00 at ηc, and 1.40 at 1.2ηc. In the matched mini-run, the idea improved the minimum real spectral margin from 0.341 to 3.295, reduced maximum Euler amplification from 0.959 to 0.606, and reduced 12-step perturbation gain from 0.334 to 0.120, but task MSE worsened from 0.000172 to 0.00561; thus the stability effect is real, but task-performance benefit was not demonstrated.
- Confidence: 8/10
- Limitations: Only one small synthetic sequence task and one fixed random seed were tested; no adding-machine benchmark, multi-seed statistics, FLOP/speed comparison, long-horizon generalization, or larger attractor-RNN evaluation was performed. The perturbation test uses a short 12-step zero-input rollout, and the spectral penalty uses a dense eigensolver.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.84895
- Paired wins: 3/8
- Benchmark verdict: no significant win

## Artifacts

- [bench_phase_margin.py](https://synthcore.org/code/1211/bench_phase_margin.py)
- [bench_report.json](https://synthcore.org/code/1211/bench_report.json)
- [experiment.py](https://synthcore.org/code/1211/experiment.py)
- [report.md](https://synthcore.org/code/1211/report.md)
- [report_bench_2026-09-03T115720.md](https://synthcore.org/code/1211/report_bench_2026-09-03T115720.md)
- [results.json](https://synthcore.org/code/1211/results.json)
- [Download all files as ZIP](https://synthcore.org/download/1211)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.