q-Boson Spectral Initialization
Implementation & benchmark of arXiv:2607.27185 — Sharp Bounds on Ground State Energy of the SYK Model
Source paper: Sharp Bounds on Ground State Energy of the SYK Model arXiv:2607.27185 ⓘ · analyzed Aug 31, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Initialize a neural layer with singular values taken from the finite spectral measure of the paper's q-boson Jacobi operator instead of using Xavier or ordinary orthogonal initialization. The resulting layer has a deliberately shaped singular-value distribution and an explicit finite-size spectral edge, allowing initialization to target stable signal propagation while retaining spectral diversity.
Formulas
Mathematical statement
The paper defines creation and annihilation operators c and c* on the basis f_0,f_1,... of ell_2(Z_{>=0}), and their symmetric sum T=c+c*. In the displayed matrix, T_{r,r+1}=T_{r+1,r}=sqrt([r+1]_q), with all other entries zero; use the q-number [r]_q=(1-q^r)/(1-q) for q != 1 and [r]_1=r. The paper's moment construction uses the quadratic form <f_0,T^{2ell}f_0>. For initialization, truncate T to indices 0,...,L-1, diagonalize T_L=U diag(lambda_1,...,lambda_L) U^T, and use the spectral measure with weights p_j=(U_{0j})^2. Its moments are m_r=e_0^T T_L^r e_0=sum_j p_j lambda_j^r. Since neural singular values are nonnegative, set s_j=alpha|lambda_j|, where alpha normalizes the second moment. The finite matrix's largest eigenvalue gives the explicit spectral edge alpha lambda_max(T_L).
Implementation notes
Integrate this at initialization of every wide MLP or projection matrix W in a transformer block, before training starts. Choose a truncation size L such as 32 or 64 and q in {0.5,0.8,0.95}. Build the symmetric tridiagonal T_L with off-diagonals sqrt((1-q^r)/(1-q)), compute its eigenpairs using a symmetric eigensolver, and form the nonnegative singular-value list s_j=alpha*abs(lambda_j). For an m-by-d layer with rank r=min(m,d), draw independent Haar orthogonal matrices U_m and V_d, select r quadrature nodes by sampling indices j with probabilities p_j=(U_T[0,j])^2, and set W=U_m diag(s_{a_1},...,s_{a_r}) V_d^T. A simpler deterministic option repeats the L-node list to fill rank r. Normalize alpha using the displayed second-moment formula, then apply ordinary residual-branch scaling separately. The paper supplies the Jacobi recurrence and moment construction; eigendecomposition, node sampling, and matrix assembly are standard numerical operations. Measure actual layer singular spectra, activation variance across depth, gradient norms, and largest singular values during the first 1,000 updates. First test a 12-layer width-256 MLP on CIFAR-10 against Xavier, orthogonal, and truncated-normal initialization at equal parameter count and optimizer settings. Success means lower activation and gradient variance, faster loss decrease, or stable training at a depth or learning rate where Xavier becomes unstable.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.