Semicircle-Calibrated Additive Initialization

(1) Integration point: use an additive block containing n square operators, such as a residual ensemble y=x+Σ_jA_jx, a multi-branch linear-attention block, or a symmetric proxy formed from each branch Jacobian. Parameterize each operator as A_j=(M_j+M_j^T)/√2 when a self-adjoint matrix is required. Apply scale calibration at initialization and optionally add the moment penalty during training. (2) Pseudocode: initialize raw matrices M_j; form A_j=(M_j+M_j^T)/√2; estimate s_j^2=d^{-1}tr(A_j^2) and m_j=d^{-1}tr|A_j|^{2+δ}; compute B=(Σ_js_j^2)^{1/2} and ε_hat=C B^{-(2+δ)}Σ_jm_j; rescale every branch by α=B_target/(B+10^{-8}), or preserve total variance while reducing branches with the largest m_j/s_j^{2+δ}. During training add λ ε_hat to the loss, optionally stopping gradients through B. (3) For small d, compute eigenvalues exactly and evaluate tr|A_j|^{2+δ}; for large d, use Hutchinson probes and Lanczos quadrature. The paper supplies the moment ratio and its decay with aggregate variance; treat C as an unknown ranking constant and set C=1 for tuning. Every few hundred steps compute S=B^{-1}Σ_jA_j, its empirical CDF, and the diagnostic Δ_hat=sup_x|F_hat_S(x)−F_ω(x)|. (4) First experiment: train a width-256, eight-branch residual MLP and a small transformer on CIFAR-10 or WikiText-2, comparing Xavier/scaled-residual initialization against this calibration, with and without the penalty. Record early loss descent, gradient-norm variance, maximum singular value, Δ_hat, and final accuracy or perplexity. Success means fewer unstable runs and lower early gradient variance at identical FLOPs; reject the method if reducing ε_hat does not correlate with better stability.