# Spherical-Design Prototype Initialization

- ID: 3056
- Canonical URL: https://synthcore.org/idea/3056/spherical-design-prototype-initialization
- API JSON: https://synthcore.org/api/idea/3056.json
- API Markdown: https://synthcore.org/api/idea/3056.md
- Verification status: unverified
- Source: [arXiv:2609.02439](https://arxiv.org/abs/2609.02439)
- Category: initialization
- Solves: accuracy, sample-efficiency, stability
- ML areas: embedding, initialization, attention, loss
- Math tags: geometry, harmonic-analysis, linear-algebra
- Ratings: usefulness 5/10; difficulty 4/10; novelty 5/10

## Idea description

Initialize directional prototypes, cosine-classifier weights, or angular attention directions with a spherical t-design rather than iid random vectors. Exact matching of spherical moments through degree t should provide uniform angular coverage and reduce initialization anisotropy, especially when the number of prototypes is small.

## Mathematical statement

A spherical t-design is a finite set X = {x_i}_{i=1}^N contained in the unit sphere S^{d-1} such that the equal-weight empirical average matches the uniform spherical probability measure sigma for every polynomial p of degree at most t: (1/N) sum_i p(x_i) = integral_{S^{d-1}} p(x) d sigma(x). Equivalently, for every tensor order m with 0 <= m <= t, (1/N) sum_i x_i^{tensor m} = E_{z sampled from sigma}[z^{tensor m}]. The first-order condition gives zero mean and the second-order condition gives isotropy, while higher orders control angular anisotropy. The paper also uses the polar vertex relation v_i = x_i/h_i, where h_i = h_{P_X}(x_i) is the support number of the associated Minkowski polytope; this provides an optional principled radial rescaling. For an ML module, use the design points as normalized prototype directions and optionally add a penalty measuring deviation from the target low-order moments.

## Key formulas

- $$\frac{1}{N}\sum_{i=1}^{N}p(x_i)=\int_{\mathbb S^{d-1}}p(x)\,d\sigma(x),\qquad \deg p\le t.$$
- $$\frac{1}{N}\sum_{i=1}^{N}x_i^{\otimes m}=\mathbb E_{z\sim\sigma}[z^{\otimes m}],\qquad 0\le m\le t.$$
- $$P_X^{\circ}=\operatorname{conv}\{x_i/h_i:1\le i\le N\},\qquad v_i=\frac{x_i}{h_i}.$$
- $$\mathcal L_{\mathrm{mom}}=\sum_{m=1}^{t}\beta_m\left\|\frac{1}{N}\sum_{i=1}^{N}z_i^{\otimes m}-M_m\right\|_F^2,\qquad M_m=\mathbb E_{z\sim\sigma}[z^{\otimes m}].$$

## Implementation notes

Use the construction in a cosine classifier, vector-quantized embedding layer, random-feature MLP, or attention head with N directional prototypes in dimension d. Obtain a spherical t-design X offline, normalize its columns, and initialize the prototype weight matrix W so that row i is x_i. For a sample embedding z, normalize it to z divided by its Euclidean norm and compute directional logits l_i = tau z^T W_i, where tau is a learnable or fixed temperature. During the first warmup period freeze W or permit only tangent-space updates: after each optimizer step replace each row w_i by w_i divided by its norm, preserving the spherical representation. If prototype adaptation is needed, add the moment regularizer L_mom using m = 1 and m = 2 first; for unit vectors the target is M_1 = 0 and M_2 = I divided by d, so the cheapest implementation is L_mom = beta_1 norm(mean_i z_i)^2 + beta_2 norm((1/N) sum_i z_i z_i^T - I/d)_F^2. The mathematical quantities are the exact design moment identities; the choice of t, beta values, temperature, and freezing schedule is empirical. To test it cheaply, train a CIFAR-10 classifier with a 16-dimensional normalized embedding and N = 64 prototypes, comparing iid Gaussian initialization, orthogonal initialization, Fibonacci-sphere initialization, and a spherical 3-design. Use identical optimizers, learning rates, augmentations, and parameter counts. Record training loss at fixed optimizer steps, final accuracy, expected calibration error, sensitivity to random seed, and empirical first- and second-moment errors. Success is lower seed variance or faster early convergence at equal compute, or equal accuracy with fewer prototypes than random initialization.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.