# Spectral-Edge Criticality Controller

- ID: 2744
- Canonical URL: https://synthcore.org/idea/2744/spectral-edge-criticality-controller
- API JSON: https://synthcore.org/api/idea/2744.json
- API Markdown: https://synthcore.org/api/idea/2744.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2608.26279](https://arxiv.org/abs/2608.26279)
- Category: dynamics
- Solves: stability, accuracy, generalization
- ML areas: rnn, ssm, training-dynamics, initialization, optimizer
- Math tags: statistical-mechanics, random-matrix, spectral-theory, dynamical-systems, bifurcations
- Ratings: usefulness 8/10; difficulty 5/10; novelty 6/10

## Idea description

Use the spectral edge of the effective recurrent Jacobian as an explicit control variable. Scale the recurrent coupling so that its largest effective eigenvalue remains a chosen distance below marginal stability, preserving long memory without allowing exploding states or gradients. The mechanism predicts a sharp change in correlation time and gradient persistence when the estimated edge crosses the critical value.

## Mathematical statement

For a symmetric Wigner coupling matrix W with entry variance \(\sigma^2/n\), the limiting eigenvalue density is \(\rho(\lambda)=\sqrt{4\sigma^2-\lambda^2}/(2\pi\sigma^2)\) on \([-2\sigma,2\sigma]\), with upper spectral edge \(\lambda_+=2\sigma\). Consider linearized recurrent dynamics \(\delta h_{t+1}=gJ_t\delta h_t\), where \(h_t\in\mathbb{R}^n\) is the hidden state, \(g>0\) is a controlled gain, and \(J_t\) is the local Jacobian excluding this gain. In the frozen symmetric approximation, marginal stability occurs at \(g\lambda_+(J)=1\), so \(g_c=1/\lambda_+(J)\), approximately \(1/(2\sigma)\) for Wigner disorder. Below this threshold, the dominant correlation decays as \(C(t)\propto(g\lambda_+)^t=\exp(-t/\xi)\), with \(\xi=-1/\log(g\lambda_+)\approx1/(1-g\lambda_+)\) near criticality. The controller maintains \(g\widehat{\lambda}_+=1-\varepsilon\), where \(\widehat{\lambda}_+\) is an online estimate and \(\varepsilon\in(0,1)\) is the target stability margin.

## Key formulas

- $$\rho(\lambda)=\frac{\sqrt{4\sigma^2-\lambda^2}}{2\pi\sigma^2},\qquad \lambda_+=2\sigma$$
- $$g_c\lambda_+(J)=1,\qquad g_c=\frac{1}{\lambda_+(J)}\approx\frac{1}{2\sigma}$$
- $$C(t)\propto(g\lambda_+)^t=\exp\!\left[-\frac{t}{\xi}\right],\qquad \xi=-\frac{1}{\log(g\lambda_+)}\approx\frac{1}{1-g\lambda_+}$$
- $$g_{t+1}=\operatorname{clip}\left(g_t\exp\left[\alpha\left((1-\varepsilon)-g_t\widehat{\lambda}_{+,t}\right)\right],g_{\min},g_{\max}\right)$$

## Implementation notes

Integrate the controller into a tanh RNN, linear state-space model, or residual sequence block immediately before recurrent multiplication: \(h_{t+1}=\phi(g_tWh_t+Ux_t+b)\). At every training step, estimate the largest singular value or largest real eigenvalue of the local Jacobian with 3--5 power iterations using a fresh Gaussian probe. For tanh, use \(J_t=D_tW\), where \(D_t=\operatorname{diag}(\phi'(g_tWh_t+Ux_t+b))\). Use the estimate \(\widehat\lambda_{+,t}\) in the multiplicative update, with stop-gradient through the estimate. Pseudocode: initialize \(g=0.9/\widehat\lambda_+\); unroll one minibatch; compute Jacobian-vector products; run power iteration; update \(g\leftarrow g\exp[\alpha((1-\varepsilon)-g\widehat\lambda_+)]\); then backpropagate task loss and update network parameters. The paper supplies the spectral-edge mechanism and critical scaling; finite-width and nonlinear Jacobian edges must be estimated empirically. First test 256-unit tanh RNNs on copy memory and sequential MNIST, comparing standard orthogonal initialization, fixed spectral normalization, and the controller with \(\varepsilon\in\{0.05,0.1,0.2,0.4\}\). Measure hidden autocorrelation, gradient norm versus sequence length, loss, and estimated edge. The quantitative prediction is a stability boundary near \(g\widehat\lambda_+=1\): below one, correlation time follows \(\xi\approx1/(1-g\widehat\lambda_+)\); above one, state or gradient variance grows rapidly. The measured critical gain should agree with \(1/\widehat\lambda_+\) within approximately 20%, and the controller should hold the edge near \(1-\varepsilon\).

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: inconclusive
- Verdict: Built a spectral-edge controller using power iteration on the local tanh Jacobian and verified the mechanism numerically. The toy sweep found the stability transition between effective edges 0.99 and 1.01, matched the predicted growth slope and correlation time to floating-point precision, and measured Wigner-edge/critical-gain scaling within 4.1%. The controller moved an initial effective edge of 2.0 to the target 0.9, but in the nonlinear recurrence it reduced gradient norm versus the fixed-gain baseline (0.00965 vs 0.219), so this is evidence for control and stability rather than a demonstrated performance win.

### Mechanism check

- Verdict: Built a spectral-edge controller using power iteration on the local tanh Jacobian and verified the mechanism numerically. The toy sweep found the stability transition between effective edges 0.99 and 1.01, matched the predicted growth slope and correlation time to floating-point precision, and measured Wigner-edge/critical-gain scaling within 4.1%. The controller moved an initial effective edge of 2.0 to the target 0.9, but in the nonlinear recurrence it reduced gradient norm versus the fixed-gain baseline (0.00965 vs 0.219), so this is evidence for control and stability rather than a demonstrated performance win.
- Confidence: 8/10
- Limitations: No copy-memory, sequential-MNIST, or trained task experiment was run; the nonlinear test used one small fixed-input tanh recurrence and a symmetric normalized matrix. The toy correlation law uses the exact dominant eigenmode, so its near-zero error validates the formula but does not establish finite-width nonlinear task benefits.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0324
- Paired wins: 7/8
- Benchmark verdict: idea better (significant)

## Artifacts

- [bench_experiment.py](https://synthcore.org/code/1056/bench_experiment.py)
- [bench_report.json](https://synthcore.org/code/1056/bench_report.json)
- [experiment.py](https://synthcore.org/code/1056/experiment.py)
- [report.md](https://synthcore.org/code/1056/report.md)
- [report_bench_2026-09-01T232430.md](https://synthcore.org/code/1056/report_bench_2026-09-01T232430.md)
- [results.json](https://synthcore.org/code/1056/results.json)
- [spectral_edge.py](https://synthcore.org/code/1056/spectral_edge.py)
- [summary.json](https://synthcore.org/code/1056/summary.json)
- [Download all files as ZIP](https://synthcore.org/download/1056)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.