# Critical Spectral Mode Compression

- ID: 1109
- Canonical URL: https://synthcore.org/idea/1109/critical-spectral-mode-compression
- API JSON: https://synthcore.org/api/idea/1109.json
- API Markdown: https://synthcore.org/api/idea/1109.md
- Verification status: mechanism_failed
- Source: [arXiv:2607.18388](https://arxiv.org/abs/2607.18388)
- Category: architecture
- Solves: memory, speedup, stability
- ML areas: ssm, rnn, inference-speedup, pruning
- Math tags: statistical-mechanics, spectral-theory, harmonic-analysis, dynamical-systems
- Ratings: usefulness 7/10; difficulty 6/10; novelty 7/10

## Idea description

Replace a large diagonalizable recurrent or state-space transition operator by a sparse set of retained oscillatory modes selected according to their contribution to the output autocorrelation. Unlike magnitude-based pruning, the objective is to preserve the power-law return signal generated by pairwise spectral differences, enabling long memory with far fewer modes.

## Mathematical statement

For a Hamiltonian eigenbasis with eigenvalues \(\varepsilon_\alpha\) and input-state weights \(w_\alpha=|\langle\varepsilon_\alpha|\nu\rangle|^2\), the return probability is \(R(\tau)=\sum_{\alpha,\beta}w_\alpha w_\beta e^{-i(\varepsilon_\alpha-\varepsilon_\beta)\tau}\). Construct a neural linear memory with complex modes \(z_j(t+1)=\lambda_j z_j(t)+b_j^\top x_t\), where \(\lambda_j=\rho_j e^{i\omega_j}\), and readout \(y_t=\sum_j c_j z_j(t)\). For stable oscillatory modes, \(\rho_j<1\). Select a subset \(S\) of modes and define normalized weights \(q_j\propto |c_jb_j|^2\). The compressed return proxy is \(R_S(k)=\sum_{j,l\in S}q_jq_l(\lambda_j\overline{\lambda_l})^k\). The mechanism predicts that if the retained frequency-spacing measure preserves the critical low-frequency pair-difference density, then \(R_S(k)\sim k^{-d_2}\) over the same intermediate-time window, even when \(|S|/D\to 0\). Stability is controlled independently by \(\max_{j\in S}|\lambda_j|<1\), while choosing \(\rho_j\) close to one creates long memory.

## Key formulas

- $$R(\tau)=\left|\langle\nu|e^{-iH\tau}|\nu\rangle\right|^2=\sum_{\alpha,\beta}w_\alpha w_\beta e^{-i(\varepsilon_\alpha-\varepsilon_\beta)\tau},\qquad w_\alpha=|\langle\varepsilon_\alpha|\nu\rangle|^2.$$
- $$z_j(t+1)=\lambda_jz_j(t)+b_j^\top x_t,\quad y_t=\sum_{j=1}^{D}c_jz_j(t),\quad |\lambda_j|<1.$$
- $$R_S(k)=\sum_{j,l\in S}q_jq_l(\lambda_j\overline{\lambda_l})^k,\qquad q_j=\frac{|c_jb_j|^2}{\sum_{m\in S}|c_mb_m|^2}.$$
- $$R_S(k)\propto k^{-d_2}\ \text{for }k_{\min}\ll k\ll k_{\max},\qquad \frac{|S|}{D}\rightarrow0\ \text{while the fitted exponent }d_2\text{ remains unchanged}.$$

## Implementation notes

1. Integration point: implement this as a replacement for the diagonal state transition in a small S4-like state-space model, diagonal linear RNN, or gated recurrent block. Begin with a teacher containing D=256 complex-conjugate stable modes and a compressed student containing M modes. 2. Pseudocode: initialize or train \(\lambda_j=\rho_j e^{i\omega_j}\), with \(0<\rho_j<1\); run the teacher on calibration sequences; estimate each mode contribution \(a_j=|c_jb_j|^2\); select modes using weighted farthest-point sampling in frequency or greedily minimize the log-scale discrepancy between teacher and student \(R(k)\) for \(k=1,\ldots,K\); renormalize \(q_j\); then fine-tune \(b\), \(c\), and retained eigenvalues using task loss plus \(\gamma\sum_k[\log(R_S(k)+\epsilon)-\log(R_D(k)+\epsilon)]^2\). Enforce \(\rho_j=\sigma(r_j)(1-\delta)\), so every mode is strictly stable. 3. Computed from the paper: the return-probability formula, pairwise spectral-difference construction, and hypothesis that sparse critical spectra preserve a power law. Estimated empirically: modal weights, exponent \(d_2\), useful time window, and modes that best preserve it. 4. First cheap experiment: train a D=256 diagonal complex SSM and compressed models with M in {8,16,32,64} on synthetic sequences generated by the teacher, Copy, and adding problems of lengths 1,000-10,000. Compare uniform frequency pruning, largest-weight pruning, and pair-difference-aware pruning. Fit \(R(k)\) on logarithmic bins. The falsifiable prediction is that M/D can fall below 0.25 while the compressed model retains the teacher's fitted exponent within 10% over at least one decade in k; random or magnitude pruning should lose the power-law window. Also test the stability boundary: a retained mode with \(|\lambda_j|\ge1\) should produce non-decaying or divergent \(R_S(k)\), whereas the constrained model should show an exponential cutoff only for \(k\gtrsim(1-\rho_{\max})^{-1}\).

## Verification

- Status: mechanism_failed
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built a reproducible spectral return-proxy MVP with stability-boundary and mode-compression sweeps. The exact single-mode prediction was confirmed: observed log-growth/decay rates matched 2 log(rho) across rho<1, rho=1, and rho>1. However, sparse frequency selection did not preserve the teacher's fitted exponent within 10% at M/D<=0.25, so the central compression mechanism was not validated.

### Mechanism check

- Verdict: Built a reproducible spectral return-proxy MVP with stability-boundary and mode-compression sweeps. The exact single-mode prediction was confirmed: observed log-growth/decay rates matched 2 log(rho) across rho<1, rho=1, and rho>1. However, sparse frequency selection did not preserve the teacher's fitted exponent within 10% at M/D<=0.25, so the central compression mechanism was not validated.
- Confidence: 9/10
- Limitations: No learned SSM, Copy or Adding task, fine-tuning, real-valued conjugate implementation, greedy pair-difference optimization, or GPU speed benchmark was tested. The finite spectrum did not exhibit a robust decade-long power law, limiting interpretation of the failed exponent-preservation test.

## Artifacts

- [REPORT.md](https://synthcore.org/code/441/REPORT.md)
- [experiment.py](https://synthcore.org/code/441/experiment.py)
- [report.md](https://synthcore.org/code/441/report.md)
- [results.json](https://synthcore.org/code/441/results.json)
- [Download all files as ZIP](https://synthcore.org/download/441)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.