# Spectral-Band Dual-Timescale Network

- ID: 1068
- Canonical URL: https://synthcore.org/idea/1068/spectral-band-dual-timescale-network
- API JSON: https://synthcore.org/api/idea/1068.json
- API Markdown: https://synthcore.org/api/idea/1068.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.17358](https://arxiv.org/abs/2607.17358)
- Category: architecture
- Solves: stability, memory, speedup
- ML areas: rnn, ssm, training-dynamics, inference-speedup
- Math tags: dynamical-systems, spectral-theory, control-theory, linear-algebra, numerical-analysis
- Ratings: usefulness 7/10; difficulty 6/10; novelty 6/10

## Idea description

Split hidden dynamics into relaxation bands when the Jacobian spectrum has a gap, evolve each band with its own timescale, and retain an explicit cross-band exchange term. This yields a principled dual-timescale RNN or SSM rather than choosing fast and slow branches heuristically.

## Mathematical statement

Let $J$ be the linearized redistribution operator and suppose its spectrum separates into fast eigenvalues at most $-\gamma_f$ and slow eigenvalues in $[-\gamma_s,0]$, with $\gamma_f\gg\gamma_s$. Let $P_f$ and $P_s$ be spectral projectors, $h_f=P_fg$, and $h_s=P_sg$. The construction applies Chapman–Enskog reduction inside each band while retaining cross-band relaxation at order one. The neural analogue uses within-band dissipative dynamics and exchange operators $E_{fs}$ and $E_{sf}$. Stability is guaranteed when the symmetric part of the full block Jacobian is negative semidefinite. The spectral gap predicts two decay timescales and a reduction breakdown when the input frequency approaches the fast relaxation rate.

## Key formulas

- $$J\approx P_fJP_f+P_sJP_s,\qquad \mathrm{spec}(P_fJP_f)\subset(-\infty,-\gamma_f],\qquad \mathrm{spec}(P_sJP_s)\subset[-\gamma_s,0],\qquad \gamma_f\gg\gamma_s.$$
- $$\dot h_f=\frac{1}{\mathrm{Da}}C_f(h_f)+E_{fs}h_s+B_fx,\qquad \dot h_s=\frac{1}{\mathrm{Da}}C_s(h_s)+E_{sf}h_f+B_sx.$$
- $$A=\begin{bmatrix}J_f/\mathrm{Da}&E_{fs}\\E_{sf}&J_s/\mathrm{Da}\end{bmatrix},\qquad \lambda_{\max}\left(\frac{A+A^\top}{2}\right)\le0\ \Longrightarrow\ \|h(t)\|\ \text{does not grow}.$$
- $$\tau_f\simeq\gamma_f^{-1},\qquad \tau_s\simeq\gamma_s^{-1},\qquad \text{reduction error increases when }\omega\tau_f\gtrsim1.$$

## Implementation notes

Implement a two-branch recurrent cell or linear/nonlinear SSM with fast state $h_f\in\mathbb{R}^{d_f}$ and slow state $h_s\in\mathbb{R}^{d_s}$. Parameterize the within-branch linear operators as $J_f=-L_f^\top L_f-\alpha_fI$ and $J_s=-L_s^\top L_s-\alpha_sI$, with trainable nonnegative $\alpha_f$ and $\alpha_s$. Add learned exchange matrices $E_{fs}$ and $E_{sf}$, then constrain the symmetric part of the complete block Jacobian by spectral normalization or projection so that $\lambda_{\max}((A+A^\top)/2)\le0$. Feed the input through separate projections $B_f$ and $B_s$. Apply the fast branch with an exact matrix exponential or a semi-implicit update, allowing larger stable macro-steps, and update the slow branch with the ordinary recurrent step. Estimate the band gap from block-Jacobian eigenvalues using a few Lanczos iterations. The paper supplies the band decomposition, within-band reduction, and order-one exchange; the neural parameterization is an implementation choice. First test on Copy Memory, sequential CIFAR-10, and synthetic signals containing two frequencies, against GRU, S4, and a single-timescale SSM. Sweep the imposed ratio $\gamma_f/\gamma_s$. Prediction: impulse responses exhibit two exponential slopes with time constants near $1/\gamma_f$ and $1/\gamma_s$; reduced dynamics becomes biased when $\omega\approx\gamma_f$, whereas retaining both bands preserves the response. Measured decay rates should agree with Jacobian eigenvalues within 20%.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: harms
- Verdict: Built and numerically verified a compact dual-timescale linear network with dissipative fast/slow branches and explicit cross-band exchange. The stability boundary matched k_crit=sqrt(gamma_f gamma_s) to 1.4e-16 relative error, measured decay-rate ratios tracked imposed spectral gaps within about 0.3%, and slow-only frequency-response error increased 3.25x near the fast relaxation rate. The dual-band representation reduced toy two-frequency reconstruction MSE by 79%, although speedup and broad task-level superiority were not tested.

### Mechanism check

- Verdict: Built and numerically verified a compact dual-timescale linear network with dissipative fast/slow branches and explicit cross-band exchange. The stability boundary matched k_crit=sqrt(gamma_f gamma_s) to 1.4e-16 relative error, measured decay-rate ratios tracked imposed spectral gaps within about 0.3%, and slow-only frequency-response error increased 3.25x near the fast relaxation rate. The dual-band representation reduced toy two-frequency reconstruction MSE by 79%, although speedup and broad task-level superiority were not tested.
- Confidence: 8/10
- Limitations: This was a small linear NumPy toy experiment rather than a trained nonlinear RNN or SSM. GRU, S4, Copy Memory, sequential CIFAR-10, inference speed, learned spectral projection, and larger-dimensional robustness were not tested; high-frequency reduction error continued rising beyond the fast-rate transition, so no sharp error peak exactly at gamma_f was established.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/431/bench_report.json)
- [report.md](https://synthcore.org/code/431/report.md)
- [report_bench_2026-08-31T120538.md](https://synthcore.org/code/431/report_bench_2026-08-31T120538.md)
- [results.json](https://synthcore.org/code/431/results.json)
- [spectral_band_experiment.py](https://synthcore.org/code/431/spectral_band_experiment.py)
- [stage2_bench.py](https://synthcore.org/code/431/stage2_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/431)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.