Initialization schemes and early-training interventions that change the loss landscape the optimizer sees.
Replace a dense token-mixing, MLP, or low-rank-adapter projection with a Bernoulli-signed sparse matrix normalized by the paper's predicted operator scale. Estimate the active representation dimension and use the effective scale to detect regimes in which extreme sparsity is likely to cause unstable amplification or dead rows.
Initialize VQ-VAE, product-quantization, or prototype embeddings from a matrix-scrambled digital net after mapping points into the data latent region. This aims to prevent early codebook collisions and dead entries by giving codewords broad coverage and controlled minimum separation, rather than relying on Gaussian initialization or random samples that contain increasingly large local gaps.
Use Marchenko–Pastur spectral edges to calibrate tensorized random features even when the base vector has exchangeable, sign-symmetric dependent coordinates. Rescale the tensor features and select their retained dimension so the predicted covariance bulk remains well-conditioned instead of assuming independent Gaussian coordinates.
Split a recurrent state into two blocks and initialize their variances and cross-correlation according to the stationary projective energy distribution induced by the transition. This places the initial hidden state near the typical invariant direction of the dynamics instead of forcing a long transient from zero or isotropic noise.
Calibrate the scales of several additive self-adjoint residual or attention operators so their aggregate spectrum has a controlled higher-moment Berry–Esseen certificate. Penalize unusually large normalized (2+δ)-moments, which should reduce spectral outliers and make the summed operator closer to a predictable semicircle-shaped spectrum.