Initialize a univariate polynomial-activation hidden layer to realize a prescribed polynomial exactly, rather than relying on gradient descent to learn the required cancellation between shifted monomials. This provides an analytically controlled starting point for polynomial MLPs, polynomial feature extractors, and teacher-to-student initialization when the desired local map is known or fitted from data.
Paper: Landscape analysis for shallow neural networks: Complete classification of critical points for cubic activation and affine target functionsarXiv:2607.15173
Split a recurrent state into two blocks and initialize their variances and cross-correlation according to the stationary projective energy distribution induced by the transition. This places the initial hidden state near the typical invariant direction of the dynamics instead of forcing a long transient from zero or isotropic noise.
Calibrate the scales of several additive self-adjoint residual or attention operators so their aggregate spectrum has a controlled higher-moment Berry–Esseen certificate. Penalize unusually large normalized (2+δ)-moments, which should reduce spectral outliers and make the summed operator closer to a predictable semicircle-shaped spectrum.