Brown-measure stability initialization
Implementation & benchmark of arXiv:2609.03618 — Spectral properties of deterministic matrices multiplied by rotationally invariant random non-Hermitian ensembles
Source paper: Spectral properties of deterministic matrices multiplied by rotationally invariant random non-Hermitian ensembles arXiv:2609.03618 ⓘ · analyzed Sep 4, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Use the paper's multiplicative spectral law to initialize a recurrent or state-space transition matrix as a deterministic operator multiplied by a unitarily invariant random factor, then choose the factor scale so the predicted complex eigenvalue support lies near a target stability annulus. This directly targets long-horizon gradient preservation instead of relying only on singular-value normalization.
Formulas
Mathematical statement
For a bi-invariant matrix A, the paper defines the radial cumulative eigenvalue distribution F_A(r)=rho_A({z in C: |z| <= r}) and states the Haagerup–Larsen relation S_{AA^*}(F_A(r)-1)=1/r^2, where rho_A is the limiting eigenvalue measure and S_{AA^*} is the scalar S-transform of the limiting eigenvalue distribution of the positive matrix AA^*. For free bi-invariant factors, the squared-singular-value S-transform of a product is the product of the factor S-transforms, so S_{(AB)(AB)^*}(z) is approximated by S_{AA^*}(z)S_{BB^*}(z). The adaptation treats a recurrent Jacobian J=AB as a large-width proxy for this law. Estimate squared-singular-value moments m_k=(1/N)tr((JJ^*)^k), form M(x)=sum_{k=1}^K m_k x^k, and define the truncated S-transform by S(z)=(1+z)M^{-1}(z)/z, where M^{-1} is the local functional inverse. Here A is the deterministic transition operator, B is a random rotationally invariant factor, J is their product, r is eigenvalue radius, and F(r) is the fraction of eigenvalues inside radius r. Select a scale alpha for B so the predicted outer radius is close to one and the predicted inner radius is not near zero.
Implementation notes
Integrate this at initialization and as an optional diagnostic for the recurrent matrix in an RNN, S4-like state-space model, or linear residual block. Factor the transition as J=A B, where A is the task-specific deterministic matrix or a learned diagonal/low-rank operator and B=U diag(s) V^T with U and V sampled as Haar-orthogonal matrices generated by QR decomposition of Gaussian matrices; include a scalar alpha multiplying B. For a calibration width, use K=6 to 10 and estimate m_k=(1/N)tr[(JJ^T)^k] with Hutchinson probes. For each random Rademacher vector v, set q_0=v and repeatedly apply q_{j+1}=JJ^Tq_j; estimate m_k by the average of v^Tq_k over probes. Fit the truncated moment polynomial M(x), numerically invert M on z in [-1,0], and solve S(F(r)-1)=1/r^2 over a radius grid. Use one-dimensional bisection on alpha so the predicted outer radius r_out is 0.95 to 0.99 for a plain RNN, or 1+epsilon for a residual or discretized state-space transition. Optionally require a predicted inner radius above 0.2 to avoid rapid mode collapse. The paper-derived quantities are the radial law and S-transform relation; moment truncation, Hutchinson estimation, and bisection are engineering approximations. Since finite-width and non-normal effects can invalidate the asymptotic proxy, also estimate the actual spectral radius with Arnoldi and the largest singular value with power iteration; if prediction error exceeds 10%, use empirical rescaling. First test a width-256 tanh RNN on copy-memory and sequential-MNIST, comparing standard orthogonal initialization, scaled Gaussian initialization, and the proposed initialization at equal parameter count. Measure gradient-norm variance versus sequence length, final accuracy, loss descent at equal FLOPs, and the actual eigenvalue cloud. Success is longer copy-memory retention and lower gradient variance without an increased largest Jacobian eigenvalue.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.