Initialization ideas

Initialization schemes and early-training interventions that change the loss landscape the optimizer sees.

Unverified 2026

Projective stationary-energy initialization

Split a recurrent state into two blocks and initialize their variances and cross-correlation according to the stationary projective energy distribution induced by the transition. This places the initial hidden state near the typical invariant direction of the dynamics instead of forcing a long transient from zero or isotropic noise.

Useful5/10
Difficulty5/10
Novelty7/10
Paper: Quantitative Furstenberg Theory for Large Random Matrices arXiv:2608.22543