Initialization ideas

Initialization schemes and early-training interventions that change the loss landscape the optimizer sees.