# Spherical covering initialization

- ID: 3045
- Canonical URL: https://synthcore.org/idea/3045/spherical-covering-initialization
- API JSON: https://synthcore.org/api/idea/3045.json
- API Markdown: https://synthcore.org/api/idea/3045.md
- Verification status: unverified
- Source: [arXiv:2609.02205](https://arxiv.org/abs/2609.02205)
- Category: architecture
- Solves: accuracy, stability, sample-efficiency
- ML areas: embedding, initialization, loss
- Math tags: geometry, metric-geometry, probability, approximation-theory
- Ratings: usefulness 5/10; difficulty 4/10; novelty 5/10

## Idea description

Replace independently random unit-normalized prototypes in a spherical classifier, vector-quantizer, or prototype contrastive head with prototypes selected to cover the sphere evenly. The method directly targets the paper's finding that weak-signal spherical K-means preserves initialization-induced Voronoi structure, reducing redundant prototypes and making early assignments less dependent on random seed.

## Mathematical statement

The paper models a directional observation v in the unit sphere \(\mathbb{S}^{d-1}=\{v\in\mathbb{R}^d:\|v\|_2=1\}\) with the von Mises-Fisher density \(f_{\mathrm{vMF}}(v;\mu,\kappa)=c_d(\kappa)\exp\{\kappa\mu^\top v\}\), where \(\mu\) is the latent unit direction, \(\kappa\) is concentration, and \(c_d(0)=1\) gives the uniform sphere. For spherical K-means with K initialized unit templates, the uniform-data population Voronoi update is governed only by their spherical Voronoi cells. The paper states that for uniformly random initializations, the average squared geodesic error after one population update is \(O(K^{-2/(d-1)})\), while the worst-case error is \(O((\log K/K)^{2/(d-1)})\). For small positive \(\kappa\), the population update differs from the \(\kappa=0\) map by \(O(\kappa)\), so initialization geometry dominates when the signal is weak. We exploit this by minimizing an empirical covering-radius proxy \(\hat r=\max_i\min_j d_{\mathrm{geo}}(x_i,p_j)\), where \(x_i\) are uniformly sampled unit vectors, \(p_j\) are prototypes, and \(d_{\mathrm{geo}}(x,p)=\arccos(x^\top p)\).

## Key formulas

- $$f_{\mathrm{vMF}}(v;\mu,\kappa)=c_d(\kappa)\exp\{\kappa\mu^{\top}v\},\qquad v,\mu\in\mathbb{S}^{d-1},\quad c_d(0)=1.$$
- $$\mathbb{E}\big[d_{\mathrm{geo}}(p_j^+,p_j)^2\big]=O\!\left(K^{-2/(d-1)}\right),\qquad \max_j d_{\mathrm{geo}}(p_j^+,p_j)^2=O\!\left(\left(\frac{\log K}{K}\right)^{2/(d-1)}\right).$$
- $$\text{population-update}_{\kappa}=\text{population-update}_{0}+O(\kappa),\qquad d_{\mathrm{geo}}(x,p)=\arccos(\operatorname{clip}(x^{\top}p,-1,1)).$$
- $$\hat r(P)=\max_{i\le M}\min_{j\le K}\arccos\!\left(\operatorname{clip}(x_i^{\top}p_j,-1,1)\right),\qquad \|p_j\|_2=1.$$

## Implementation notes

Integrate this at initialization of the prototype matrix in a spherical classifier, spherical vector-quantizer, or contrastive prototype head. First draw M unit vectors x_i by sampling z_i\sim\mathcal N(0,I_d) and setting x_i=z_i/\|z_i\|_2; use M between 20K and 100K. Select p_1=x_1, then repeatedly choose p_j as the x_i maximizing its current distance to the selected set: dist_i=\min_{l<j}\arccos(\operatorname{clip}(x_i^    op p_l,-1,1)); set p_j=x_{\arg\max_i dist_i}. This farthest-point greedy procedure minimizes the empirical covering-radius proxy \hat r and approximates an even spherical Voronoi tessellation. For additional refinement, run 5-20 Lloyd iterations on the sampled sphere: assign each x_i to the prototype with largest dot product, replace p_j by the normalized sum of its assigned samples, and renormalize. Use the resulting P as the trainable prototype initialization. The paper supplies the random-initialization covering-rate scalings and the O(\kappa) weak-signal perturbation; the implementation estimates \hat r empirically and does not require estimating \kappa. First test on synthetic vMF mixtures in d=32 or d=128 with K=32,64,128, followed by CIFAR-10 embeddings. Compare random-normalized, orthogonal, and covering initialization over five seeds. Record covering radius, assignment entropy, dead-prototype fraction, clustering accuracy, final loss, and seed-to-seed variance. Success means lower covering radius, fewer dead or redundant prototypes, and better weak-signal accuracy without degradation for strongly concentrated mixtures.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.