# Transfer-Spectrum Pseudo-Transition Scheduler

- ID: 1234
- Canonical URL: https://synthcore.org/idea/1234/transfer-spectrum-pseudo-transition-scheduler
- API JSON: https://synthcore.org/api/idea/1234.json
- API Markdown: https://synthcore.org/api/idea/1234.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.21359](https://arxiv.org/abs/2607.21359)
- Category: dynamics
- Solves: stability, speed, accuracy
- ML areas: training-dynamics, optimizer, rnn, ssm
- Math tags: statistical-mechanics, dynamical-systems, spectral-theory, linear-algebra, bifurcations
- Ratings: usefulness 7/10; difficulty 6/10; novelty 7/10

## Idea description

Represent the propagation of hidden states, layer states, or optimizer states by a locally estimated transfer operator and monitor its leading eigenvalue gap. When two dominant modes undergo an avoided crossing, reduce the update scale or increase damping; after the gap reopens, restore the normal schedule. This imports the paper's sharp-but-continuous pseudo-transition mechanism rather than treating instability as a binary divergence event.

## Mathematical statement

Let T(theta) be an effective transfer operator, where theta is a training, layer, or input-control parameter, and let lambda_0, lambda_1, and lambda_2 be its eigenvalues ordered by magnitude. The free-energy analogue is f(theta) = -log(lambda_0(theta)), and the spectral correlation length is xi(theta) = [log(lambda_0(theta) / abs(lambda_1(theta)))]^{-1}. If two uncoupled modes have eigenvalues a(theta) and b(theta), a finite coupling g creates an avoided crossing with eigenvalues lambda_plus and lambda_minus. The response becomes largest near a(theta) = b(theta), while long-range memory can instead be controlled by another subleading eigenvalue. The scheduler uses the normalized gap G and the projection entropy H to detect this pseudo-transition.

## Key formulas

- $$f(\theta)=-\log \lambda_0(\theta),\qquad \xi(\theta)=\left[\log\frac{\lambda_0(\theta)}{|\lambda_1(\theta)|}\right]^{-1}.$$
- $$\lambda_{\pm}(\theta)=\frac{a(\theta)+b(\theta)}{2}\pm\sqrt{\left(\frac{a(\theta)-b(\theta)}{2}\right)^2+g^2}.$$
- $$G_t=\frac{\lambda_{0,t}-|\lambda_{1,t}|}{\lambda_{0,t}},\qquad H_t=-\sum_k p_{k,t}\log\left(p_{k,t}+\varepsilon\right).$$
- $$\eta_{t+1}=\eta_t\,\operatorname{clip}\left(\frac{G_t}{G_{\mathrm{target}}},r_{\min},r_{\max}\right),\qquad \eta_{t+1}\leftarrow\gamma\eta_{t+1}\ \text{if}\ H_t>H_{\mathrm{trigger}}\ \text{and}\ G_t<G_{\mathrm{trigger}}.$$

## Implementation notes

Integrate this into an optimizer for a small tanh RNN, state-space model, or deep residual network. Treat a minibatch of consecutive hidden states h_t as samples from a local propagation process. Every K optimization steps, estimate a low-rank transfer operator T by fitting h_{t+1} approximately equal to T h_t for linear models, or by using Jacobian-vector products J_t v = partial h_{t+1} / partial h_t times v and randomized Arnoldi iteration for nonlinear models. Compute the two or three eigenvalues with largest magnitude, normalize by the largest magnitude, and estimate p_k from projections of the current state batch onto the corresponding eigenvectors. Pseudocode: collect hidden states; run m power or Arnoldi iterations; obtain lambda_0, lambda_1, and lambda_2; calculate G, H, and xi; if G is below its trigger and H is above its trigger, multiply the learning rate by gamma = 0.5 and optionally increase weight decay; otherwise update the learning rate using the gap controller; then perform the ordinary gradient step. The transfer-matrix interpretation is taken from the paper, whereas T, eigenvalues, and thresholds are estimated empirically. First test on a one-layer tanh RNN trained on permuted MNIST or the adding problem, comparing Adam, fixed-step SGD, and the scheduler. Sweep learning rates and locate the largest stable rate. The quantitative prediction is that gradient variance and hidden-state sensitivity peak in a narrow band where G is minimized and H rises. Damping should reduce this peak without eliminating the surrounding stable regimes. Across seeds, the measured response peak should coincide with the minimum-gap location within 20 percent, and the measured memory decay should follow C(k) proportional to exp(-k / xi) using the estimated subleading eigenvalue.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built a compact transfer-spectrum scheduler prototype and an exact avoided-crossing verification in experiment.py. Across g=0.02, 0.05, 0.10, and 0.20, the measured gap/curvature response was centered at theta=0, the splitting matched 2g exactly, and xi matched the predicted base/(2g) within about 1.4%; the independently fitted memory decay also matched xi. In the small tanh-RNN adding-task comparison, the scheduler never triggered and baseline and idea were identical, so there is no observed practical training win; worked=true reflects the requested mechanism validation, not optimizer improvement.

### Mechanism check

- Verdict: Built a compact transfer-spectrum scheduler prototype and an exact avoided-crossing verification in experiment.py. Across g=0.02, 0.05, 0.10, and 0.20, the measured gap/curvature response was centered at theta=0, the splitting matched 2g exactly, and xi matched the predicted base/(2g) within about 1.4%; the independently fitted memory decay also matched xi. In the small tanh-RNN adding-task comparison, the scheduler never triggered and baseline and idea were identical, so there is no observed practical training win; worked=true reflects the requested mechanism validation, not optimizer improvement.
- Confidence: 8/10
- Limitations: Only an analytic two-mode toy transfer operator and one small one-seed RNN adding-task run were tested. The proposed scheduler was not stress-tested across learning rates or seeds, did not demonstrate damping or stability gains, and its entropy trigger remained inactive because the fitted RNN transfer spectrum was strongly single-mode.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_experiment.py](https://synthcore.org/code/496/bench_experiment.py)
- [bench_report.json](https://synthcore.org/code/496/bench_report.json)
- [experiment.py](https://synthcore.org/code/496/experiment.py)
- [report.md](https://synthcore.org/code/496/report.md)
- [report_bench_2026-08-31T161455.md](https://synthcore.org/code/496/report_bench_2026-08-31T161455.md)
- [results.json](https://synthcore.org/code/496/results.json)
- [Download all files as ZIP](https://synthcore.org/download/496)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.