✗ Mechanism failed 2026

Critical Spectral Mode Compression

Implementation & benchmark of arXiv:2607.18388 — Information Compression at Criticality

Usefulness7/10

Difficulty6/10

Novelty7/10

Architecture Memory Speedup Stability Ssm Rnn Inference speedup Pruning Statistical mechanics Spectral theory Harmonic analysis Dynamical systems

Source paper: Information Compression at Criticality arXiv:2607.18388 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace a large diagonalizable recurrent or state-space transition operator by a sparse set of retained oscillatory modes selected according to their contribution to the output autocorrelation. Unlike magnitude-based pruning, the objective is to preserve the power-law return signal generated by pairwise spectral differences, enabling long memory with far fewer modes.

Formulas

$$R(\tau)=\left|\langle\nu|e^{-iH\tau}|\nu\rangle\right|^2=\sum_{\alpha,\beta}w_\alpha w_\beta e^{-i(\varepsilon_\alpha-\varepsilon_\beta)\tau},\qquad w_\alpha=|\langle\varepsilon_\alpha|\nu\rangle|^2.$$

$$z_j(t+1)=\lambda_jz_j(t)+b_j^\top x_t,\quad y_t=\sum_{j=1}^{D}c_jz_j(t),\quad |\lambda_j|<1.$$

$$R_S(k)=\sum_{j,l\in S}q_jq_l(\lambda_j\overline{\lambda_l})^k,\qquad q_j=\frac{|c_jb_j|^2}{\sum_{m\in S}|c_mb_m|^2}.$$

$$R_S(k)\propto k^{-d_2}\ \text{for }k_{\min}\ll k\ll k_{\max},\qquad \frac{|S|}{D}\rightarrow0\ \text{while the fitted exponent }d_2\text{ remains unchanged}.$$

Mathematical statement

For a Hamiltonian eigenbasis with eigenvalues $\varepsilon_\alpha$ and input-state weights $w_\alpha=|\langle\varepsilon_\alpha|\nu\rangle|^2$, the return probability is $R(\tau)=\sum_{\alpha,\beta}w_\alpha w_\beta e^{-i(\varepsilon_\alpha-\varepsilon_\beta)\tau}$. Construct a neural linear memory with complex modes $z_j(t+1)=\lambda_j z_j(t)+b_j^\top x_t$, where $\lambda_j=\rho_j e^{i\omega_j}$, and readout $y_t=\sum_j c_j z_j(t)$. For stable oscillatory modes, $\rho_j<1$. Select a subset $S$ of modes and define normalized weights $q_j\propto |c_jb_j|^2$. The compressed return proxy is $R_S(k)=\sum_{j,l\in S}q_jq_l(\lambda_j\overline{\lambda_l})^k$. The mechanism predicts that if the retained frequency-spacing measure preserves the critical low-frequency pair-difference density, then $R_S(k)\sim k^{-d_2}$ over the same intermediate-time window, even when $|S|/D\to 0$. Stability is controlled independently by $\max_{j\in S}|\lambda_j|<1$, while choosing $\rho_j$ close to one creates long memory.

Implementation notes

1. Integration point: implement this as a replacement for the diagonal state transition in a small S4-like state-space model, diagonal linear RNN, or gated recurrent block. Begin with a teacher containing D=256 complex-conjugate stable modes and a compressed student containing M modes. 2. Pseudocode: initialize or train $\lambda_j=\rho_j e^{i\omega_j}$, with $0<\rho_j<1$; run the teacher on calibration sequences; estimate each mode contribution $a_j=|c_jb_j|^2$; select modes using weighted farthest-point sampling in frequency or greedily minimize the log-scale discrepancy between teacher and student $R(k)$ for $k=1,\ldots,K$; renormalize $q_j$; then fine-tune $b$, $c$, and retained eigenvalues using task loss plus $\gamma\sum_k[\log(R_S(k)+\epsilon)-\log(R_D(k)+\epsilon)]^2$. Enforce $\rho_j=\sigma(r_j)(1-\delta)$, so every mode is strictly stable. 3. Computed from the paper: the return-probability formula, pairwise spectral-difference construction, and hypothesis that sparse critical spectra preserve a power law. Estimated empirically: modal weights, exponent $d_2$, useful time window, and modes that best preserve it. 4. First cheap experiment: train a D=256 diagonal complex SSM and compressed models with M in {8,16,32,64} on synthetic sequences generated by the teacher, Copy, and adding problems of lengths 1,000-10,000. Compare uniform frequency pruning, largest-weight pruning, and pair-difference-aware pruning. Fit $R(k)$ on logarithmic bins. The falsifiable prediction is that M/D can fall below 0.25 while the compressed model retains the teacher's fitted exponent within 10% over at least one decade in k; random or magnitude pruning should lose the power-law window. Also test the stability boundary: a retained mode with $|\lambda_j|\ge1$ should produce non-decaying or divergent $R_S(k)$, whereas the constrained model should show an exponential cutoff only for $k\gtrsim(1-\rho_{\max})^{-1}$.

Verification

Mechanism failed

Stage 1 · Toy mechanism gate: Failed ✗

Stage 2 · Mechanism transferred to benchmark: Not tested

Stage 2 · Practical benchmark result: Not run

Methodology: Toy-system gate first; the benchmark stage runs only after a pass. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built a reproducible spectral return-proxy MVP with stability-boundary and mode-compression sweeps. The exact single-mode prediction was confirmed: observed log-growth/decay rates matched 2 log(rho) across rho<1, rho=1, and rho>1. However, sparse frequency selection did not preserve the teacher's fitted exponent within 10% at M/D<=0.25, so the central compression mechanism was not validated.

Agent confidence: 9/10

Baseline: At M=32/64, random selection log-RMSE was 3.391/2.743 and exponent relative error was 102%/69%.
Idea: Frequency-quantile selection at M=32/64 had log-RMSE 3.447/2.740 and exponent relative error 84%/38%; mode-scaling log-RMSE decreased from 5.37 at M=4 to 1.80 at M=128, but remained poor under 25% retention.

Limitations: No learned SSM, Copy or Adding task, fine-tuning, real-valued conjugate implementation, greedy pair-difference optimization, or GPU speed benchmark was tested. The finite spectrum did not exhibit a robust decade-long power law, limiting interpretation of the failed exponent-preservation test.

How to run: python3 experiment.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Artifacts

Implementation overview ⬇ Download all as ZIP 4 files · code, reports and structured results

📄 REPORT.md 2.6 KB View
🐍 experiment.py 3.8 KB View Raw
📄 report.md 1.3 KB View
📊 results.json 4.8 KB View Raw