# Sideband-Aware Stability Monitor for Periodic Training

- ID: 628
- Canonical URL: https://synthcore.org/idea/628/sideband-aware-stability-monitor-for-periodic-training
- API JSON: https://synthcore.org/api/idea/628.json
- API Markdown: https://synthcore.org/api/idea/628.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.09275](https://arxiv.org/abs/2607.09275)
- Category: dynamics
- Solves: stability, accuracy
- ML areas: optimizer, rnn, ssm, training-dynamics
- Math tags: dynamical-systems, control-theory, spectral-theory, harmonic-analysis, linear-algebra
- Ratings: usefulness 7/10; difficulty 6/10; novelty 7/10

## Idea description

Replace the usual averaged Jacobian test for a periodically modulated neural update with a finite harmonic-transfer model that explicitly couples perturbation frequencies separated by the modulation frequency. Use the resulting lifted spectral radius to cap the learning rate or reduce modulation amplitude when sideband interactions create an instability that is invisible in the averaged model.

## Mathematical statement

Linearize one period of a periodically modulated training or recurrent update as \(\dot{\delta x}=A(t)\delta x\), where \(\delta x\) is a parameter or state perturbation and \(A(t)=\sum_{m=-M}^{M}A_m e^{im\omega_s t}\) has period \(T_s=2\pi/\omega_s\). A perturbation at frequency \(\omega\) has Fourier components \(\delta x_k\) at sidebands \(\omega+k\omega_s\), and harmonic balance gives \(j(\omega+k\omega_s)\delta x_k=\sum_m A_m\delta x_{k-m}\). Truncating to \(k\in[-K,K]\) produces a lifted matrix whose spectral radius or resolvent gain determines amplification. For a discrete update, stability requires \(\rho(\mathcal{M})<1\), where \(\mathcal{M}=\prod_{r=0}^{S-1}(I+\eta A_r)\) is the one-period monodromy matrix, \(\eta\) is the base step size, and \(A_r\) is the Jacobian at phase \(r\). Retaining off-diagonal sideband blocks can reveal instability near or above half the modulation frequency that is absent from the averaged model.

## Key formulas

- $$A(t)=\sum_{m=-M}^{M}A_m e^{im\omega_s t},\qquad \delta x(t)=\sum_{k=-K}^{K}\delta x_k e^{j(\omega+k\omega_s)t}$$
- $$j(\omega+k\omega_s)\delta x_k=\sum_{m=-M}^{M}A_m\delta x_{k-m},\qquad \mathcal{A}_{k\ell}(\omega)=A_{k-\ell}-j(\omega+k\omega_s)I\,\mathbf{1}_{k=\ell}$$
- $$\mathcal{G}(\omega)=\left(j\Omega-\mathcal{A}_0-\mathcal{C}\right)^{-1},\qquad \rho(\mathcal{M})<1,\quad \mathcal{M}=\prod_{r=0}^{S-1}\left(I+\eta A_r\right)$$
- $$\eta_{c,\mathrm{lift}}=\sup\{\eta:\rho(\mathcal{M}(\eta))<1\},\qquad \eta_{c,\mathrm{avg}}\text{ is obtained by replacing }A_r\text{ with }\bar A=S^{-1}\sum_r A_r$$

## Implementation notes

1. Integration point: apply this to an optimizer with a periodic learning-rate, momentum, gradient-clipping, or data-order schedule, or to an SSM/RNN whose recurrent matrix is periodically gated. Every \(S\) optimizer steps, estimate the local parameter-space Jacobian at each schedule phase. For a cheap MVP, use a two-layer MLP on MNIST with cosine learning-rate modulation of periods \(S=8,16,32\), and compare ordinary SGD or momentum SGD with the sideband-aware controller. 2. Pseudocode: for each phase \(r=0,\ldots,S-1\), compute Jacobian-vector products \(A_rv\) with autograd at the current minibatch or a fixed probe batch; apply the phase operators successively to Lanczos or power-iteration vectors using \(v\leftarrow v+\eta A_rv\); estimate \(\rho(\mathcal M)\). If the estimate exceeds \(1-\epsilon\), shrink \(\eta\leftarrow0.8\eta\) or reduce modulation amplitude. Optionally estimate Fourier coefficients \(A_m=S^{-1}\sum_r A_re^{-2\pi imr/S}\) and construct the block sideband matrix with \(K=1\) or \(2\). 3. Computed versus empirical: monodromy and lifted operators are computed from Jacobian-vector products; minibatch noise and nonlinear basin effects are measured empirically. The averaged baseline uses only \(\bar A=S^{-1}\sum_rA_r\), deleting off-diagonal sideband blocks. 4. First experiment: train the same MLP on MNIST across learning-rate sweeps and modulation periods \(S=8,16,32\), recording loss divergence, gradient norm, \(\rho(\mathcal M)\), and the averaged-Jacobian prediction. The falsifiable prediction is that the actual divergence threshold follows \(\rho(\mathcal M)=1\) within 20%, while the averaged model increasingly overestimates the safe learning rate when modulation amplitude is large or perturbation content is near the schedule's Nyquist frequency. A second signature is a sharp rise in sideband gain near \(\omega\approx\omega_s/2\) or its aliases.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: harms
- Verdict: Built a reproducible periodic two-phase shear MVP with exact monodromy, averaged-Jacobian prediction, empirical perturbation growth, Fourier coefficients, and a finite lifted-model prototype. The mechanism manifested: at g=0 exact and averaged models agree; for g=0.05–0.5 the rho gap at eta=0.3 scales quadratically, with gap/g^2=0.0227–0.0249; at g=1.5 the exact stability boundary is eta≈0.50 versus eta≈1.1429 for the averaged model, and eta=0.6 is exact-unstable but averaged-stable. This supports the sideband/noncommutativity claim in the toy setting, but is not evidence of a neural-network training win.

### Mechanism check

- Verdict: Built a reproducible periodic two-phase shear MVP with exact monodromy, averaged-Jacobian prediction, empirical perturbation growth, Fourier coefficients, and a finite lifted-model prototype. The mechanism manifested: at g=0 exact and averaged models agree; for g=0.05–0.5 the rho gap at eta=0.3 scales quadratically, with gap/g^2=0.0227–0.0249; at g=1.5 the exact stability boundary is eta≈0.50 versus eta≈1.1429 for the averaged model, and eta=0.6 is exact-unstable but averaged-stable. This supports the sideband/noncommutativity claim in the toy setting, but is not evidence of a neural-network training win.
- Confidence: 8/10
- Limitations: Only a 2D synthetic linear periodic system was tested; no MNIST/MLP, optimizer controller, JVP estimation, CUDA run, FLOP/speed comparison, noise robustness, or period/frequency sweep was performed. The prototype lifted matrix is poorly normalized in this discrete formulation and returned rho≈1 across modulation values, so the validated result is the exact monodromy versus averaged model rather than the lifted spectral-radius implementation.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.03245
- Paired wins: 2/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/281/bench_report.json)
- [bench_sideband.py](https://synthcore.org/code/281/bench_sideband.py)
- [report.md](https://synthcore.org/code/281/report.md)
- [report_bench_2026-08-31T000703.md](https://synthcore.org/code/281/report_bench_2026-08-31T000703.md)
- [results.json](https://synthcore.org/code/281/results.json)
- [sideband_monitor.py](https://synthcore.org/code/281/sideband_monitor.py)
- [Download all files as ZIP](https://synthcore.org/download/281)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.