# Volterra BBP multi-timescale recurrent initialization

- ID: 692
- Canonical URL: https://synthcore.org/idea/692/volterra-bbp-multi-timescale-recurrent-initialization
- API JSON: https://synthcore.org/api/idea/692.json
- API Markdown: https://synthcore.org/api/idea/692.md
- Verification status: mechanism_failed
- Source: [arXiv:2607.10503](https://arxiv.org/abs/2607.10503)
- Category: architecture
- Solves: stability, accuracy, sample-efficiency
- ML areas: rnn, ssm, initialization, world-model
- Math tags: random-matrix, spectral-theory, dynamical-systems, linear-algebra
- Ratings: usefulness 7/10; difficulty 5/10; novelty 8/10

## Idea description

Initialize a recurrent or state-space transition matrix with weak Wigner noise plus a shared cumulative-sum correlation structure. Increasing the correlation strength produces recurrent eigenmodes one at a time at analytically predicted BBP thresholds, yielding a controllable hierarchy of short- and long-memory modes. The matrix should then be globally rescaled or constrained so that all active modes remain inside the desired stability radius.

## Mathematical statement

Let $G$ be a symmetric Wigner matrix with independent centered unit-variance entries, let $Z_1,\ldots,Z_N$ be independent centered unit-variance factors, and define the symmetric shared-factor matrix $M_{ij}=Z_{\max(i,j)}$ for $i\neq j$ and $M_{ii}=Z_i$. Construct $W_0=(\sigma/\sqrt{N})G+(b/N)M$, where $\sigma>0$ is the bulk-noise scale and $b\geq 0$ is the correlation strength. In the continuum limit, the correlated eigenvalues are governed by the singular values $s_k$ of the Volterra operator $(Vf)(t)=\int_0^t f(u)\,du$ on $[0,1]$, with $s_k=1/((k-1/2)\pi)$. Equivalently, the eigenvalues of $V^*V$ are $s_k^2=1/((k-1/2)^2\pi^2)$. The $k$th BBP transition occurs approximately when $b s_k=\sigma$, and above threshold the associated outlier is approximately $\lambda_k=b s_k+\sigma^2/(b s_k)$, outside the Wigner bulk edge $2\sigma$. For a recurrent update $h_{t+1}=\phi(Wh_t+Ux_t)$, a global gain can enforce a target spectral radius below one while preserving the relative outlier hierarchy.

## Key formulas

- $$W_0=\frac{\sigma}{\sqrt{N}}G+\frac{b}{N}M,\qquad M_{ij}=Z_{\max(i,j)},\qquad G=G^{\mathsf T}.$$
- $$(Vf)(t)=\int_0^t f(u)\,du,\qquad s_k(V)=\frac{1}{(k-\tfrac12)\pi},\qquad s_k(V^*V)=\frac{1}{(k-\tfrac12)^2\pi^2}.$$
- $$b_{c,k}s_k=\sigma,\qquad \lambda_k^{\mathrm{out}}\approx b s_k+\frac{\sigma^2}{b s_k}\quad\text{for }b s_k>\sigma.$$
- $$h_{t+1}=\phi(gW_0h_t+Ux_t),\qquad g=\frac{r}{\max_j|\lambda_j(W_0)|},\quad 0<r<1.$$

## Implementation notes

First, integrate the construction at the recurrent transition matrix of a tanh RNN, a linear state-space model, or a recurrent world model. Sample a symmetric Wigner matrix $G$ and a vector $Z$ of independent standard normal variables. Form $M[i,j]=Z[\max(i,j)]$, symmetrize it, and set $W_0=\sigma G/\sqrt{N}+bM/N$. Estimate the largest eigenvalue using 10 to 30 power iterations, then rescale with $g=r/(|\lambda_{\max}(W_0)|+10^{-6})$, where $r$ is typically 0.9 or 0.95. The forward update is $h=\tanh(gW_0h+Ux)$. Optionally make $b$ trainable and add the penalty $\alpha\max(0,\rho(W)-r)^2$, estimating $\rho(W)$ by power iteration. The paper supplies the Volterra singular values and predicted thresholds $b_{c,k}=\sigma/s_k$; the implementation estimates finite-size eigenvalues, actual nonlinear Jacobian growth, and task performance. Run a cheap experiment with $N=256$ tanh RNNs on copy-memory and adding tasks, comparing this initialization against iid Gaussian and orthogonal initialization with matched optimizer settings. For $\sigma=0.5$, sweep $b$ across the first several predicted thresholds and diagonalize $W_0$. The falsifiable prediction is a staircase: the number of eigenvalues outside the bulk edge $2\sigma$ should increase near $b_{c,k}=\sigma/s_k$, with the first few transition locations within about 20 percent at finite $N$. Perturbations projected onto each outlier eigenvector should decay approximately as $|g\lambda_k|^t$, producing distinct memory timescales.

## Verification

- Status: mechanism_failed
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built and tested the Volterra BBP initialization, including matrix construction, spectral staircase sweeps, outlier asymptotic checks, global gain rescaling, and an independent Volterra discretization. The analytic Volterra singular values matched to 0.10%, and rescaling produced the requested spectral radius 0.900 versus 0.9. However, the finite-size first transition occurred at b=0.609 versus predicted 0.785 (22.5% relative error), while the outlier formula errors were 1.4% for mode 1 but 127% and 282% for modes 2 and 3; therefore the claimed multi-timescale BBP mechanism was not confirmed.

### Mechanism check

- Verdict: Built and tested the Volterra BBP initialization, including matrix construction, spectral staircase sweeps, outlier asymptotic checks, global gain rescaling, and an independent Volterra discretization. The analytic Volterra singular values matched to 0.10%, and rescaling produced the requested spectral radius 0.900 versus 0.9. However, the finite-size first transition occurred at b=0.609 versus predicted 0.785 (22.5% relative error), while the outlier formula errors were 1.4% for mode 1 but 127% and 282% for modes 2 and 3; therefore the claimed multi-timescale BBP mechanism was not confirmed.
- Confidence: 9/10
- Limitations: The planned tanh copy-memory/adding-task comparison was not run after the spectral mechanism failed; experiments used a reduced N=96 and 8 or fewer repeats because the original N=256 sweep exceeded the execution budget. The implemented M is the stated random max-index factor matrix, but its finite-dimensional spectral structure was not theoretically re-derived beyond the supplied claim.

## Artifacts

- [report.md](https://synthcore.org/code/299/report.md)
- [results.json](https://synthcore.org/code/299/results.json)
- [verify_volterra.py](https://synthcore.org/code/299/verify_volterra.py)
- [volterra_bbp_experiment.py](https://synthcore.org/code/299/volterra_bbp_experiment.py)
- [volterra_singular_values.json](https://synthcore.org/code/299/volterra_singular_values.json)
- [Download all files as ZIP](https://synthcore.org/download/299)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.