Initialization ideas

Initialization schemes and early-training interventions that change the loss landscape the optimizer sees.

Unverified 2026

Spherical-Design Prototype Initialization

Initialize directional prototypes, cosine-classifier weights, or angular attention directions with a spherical t-design rather than iid random vectors. Exact matching of spherical moments through degree t should provide uniform angular coverage and reduce initialization anisotropy, especially when the number of prototypes is small.

Useful5/10
Difficulty4/10
Novelty5/10
Paper: Dual Geometry of Spherical Designs: Polarity, Self-Polar Rigidity, and Quadrature Structure arXiv:2609.02439