# Brown-measure stability initialization

- ID: 3141
- Canonical URL: https://synthcore.org/idea/3141/brown-measure-stability-initialization
- API JSON: https://synthcore.org/api/idea/3141.json
- API Markdown: https://synthcore.org/api/idea/3141.md
- Verification status: unverified
- Source: [arXiv:2609.03618](https://arxiv.org/abs/2609.03618)
- Category: dynamics
- Solves: stability, accuracy, sample-efficiency
- ML areas: rnn, ssm, initialization, training-dynamics
- Math tags: random-matrix, spectral-theory, probability, linear-algebra
- Ratings: usefulness 6/10; difficulty 6/10; novelty 6/10

## Idea description

Use the paper's multiplicative spectral law to initialize a recurrent or state-space transition matrix as a deterministic operator multiplied by a unitarily invariant random factor, then choose the factor scale so the predicted complex eigenvalue support lies near a target stability annulus. This directly targets long-horizon gradient preservation instead of relying only on singular-value normalization.

## Mathematical statement

For a bi-invariant matrix A, the paper defines the radial cumulative eigenvalue distribution F_A(r)=rho_A({z in C: |z| <= r}) and states the Haagerup–Larsen relation S_{AA^*}(F_A(r)-1)=1/r^2, where rho_A is the limiting eigenvalue measure and S_{AA^*} is the scalar S-transform of the limiting eigenvalue distribution of the positive matrix AA^*. For free bi-invariant factors, the squared-singular-value S-transform of a product is the product of the factor S-transforms, so S_{(AB)(AB)^*}(z) is approximated by S_{AA^*}(z)S_{BB^*}(z). The adaptation treats a recurrent Jacobian J=AB as a large-width proxy for this law. Estimate squared-singular-value moments m_k=(1/N)tr((JJ^*)^k), form M(x)=sum_{k=1}^K m_k x^k, and define the truncated S-transform by S(z)=(1+z)M^{-1}(z)/z, where M^{-1} is the local functional inverse. Here A is the deterministic transition operator, B is a random rotationally invariant factor, J is their product, r is eigenvalue radius, and F(r) is the fraction of eigenvalues inside radius r. Select a scale alpha for B so the predicted outer radius is close to one and the predicted inner radius is not near zero.

## Key formulas

- $$F_{\mathbf{A}}(r)=\rho_{\mathbf{A}}\big(\{z\in\mathbb{C}:|z|\le r\}\big)$$
- $$S_{\mathbf{A}\mathbf{A}^{*}}\big(F_{\mathbf{A}}(r)-1\big)=\frac{1}{r^{2}}$$
- $$S_{(\mathbf{A}\mathbf{B})(\mathbf{A}\mathbf{B})^{*}}(z)\approx S_{\mathbf{A}\mathbf{A}^{*}}(z)S_{\mathbf{B}\mathbf{B}^{*}}(z)$$
- $$\widehat{S}(z)=\frac{1+z}{z}\,\widehat{M}^{-1}(z),\qquad \widehat{M}(x)=\sum_{k=1}^{K}\widehat{m}_{k}x^{k},\qquad \widehat{m}_{k}=\frac{1}{N}\operatorname{tr}\!\left[(JJ^{*})^{k}\right]$$

## Implementation notes

Integrate this at initialization and as an optional diagnostic for the recurrent matrix in an RNN, S4-like state-space model, or linear residual block. Factor the transition as J=A B, where A is the task-specific deterministic matrix or a learned diagonal/low-rank operator and B=U diag(s) V^T with U and V sampled as Haar-orthogonal matrices generated by QR decomposition of Gaussian matrices; include a scalar alpha multiplying B. For a calibration width, use K=6 to 10 and estimate m_k=(1/N)tr[(JJ^T)^k] with Hutchinson probes. For each random Rademacher vector v, set q_0=v and repeatedly apply q_{j+1}=JJ^Tq_j; estimate m_k by the average of v^Tq_k over probes. Fit the truncated moment polynomial M(x), numerically invert M on z in [-1,0], and solve S(F(r)-1)=1/r^2 over a radius grid. Use one-dimensional bisection on alpha so the predicted outer radius r_out is 0.95 to 0.99 for a plain RNN, or 1+epsilon for a residual or discretized state-space transition. Optionally require a predicted inner radius above 0.2 to avoid rapid mode collapse. The paper-derived quantities are the radial law and S-transform relation; moment truncation, Hutchinson estimation, and bisection are engineering approximations. Since finite-width and non-normal effects can invalidate the asymptotic proxy, also estimate the actual spectral radius with Arnoldi and the largest singular value with power iteration; if prediction error exceeds 10%, use empirical rescaling. First test a width-256 tanh RNN on copy-memory and sequential-MNIST, comparing standard orthogonal initialization, scaled Gaussian initialization, and the proposed initialization at equal parameter count. Measure gradient-norm variance versus sequence length, final accuracy, loss descent at equal FLOPs, and the actual eigenvalue cloud. Success is longer copy-memory retention and lower gradient variance without an increased largest Jacobian eigenvalue.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.