# Spherical harmonic spectrum regularizer

- ID: 2726
- Canonical URL: https://synthcore.org/idea/2726/spherical-harmonic-spectrum-regularizer
- API JSON: https://synthcore.org/api/idea/2726.json
- API Markdown: https://synthcore.org/api/idea/2726.md
- Verification status: mechanism_failed
- Source: [arXiv:2608.24867](https://arxiv.org/abs/2608.24867)
- Category: regularization
- Solves: generalization, stability, accuracy
- ML areas: embedding, regularization, optimizer
- Math tags: harmonic-analysis, geometry, optimization, dynamical-systems
- Ratings: usefulness 7/10; difficulty 5/10; novelty 7/10

## Idea description

Constrain a set of learnable or batch-produced unit-norm embeddings by matching their spherical-harmonic power spectrum to a target spectrum rather than relying only on pairwise Euclidean repulsion. This creates an explicit, tunable mechanism for suppressing low-frequency density fluctuations or enhancing a selected angular frequency, which can improve uniformity and reduce representation collapse on hyperspherical embeddings.

## Mathematical statement

For unit vectors $u_i\in\mathbb S^2$ with optional weights $c_i$ satisfying $\sum_i c_i=N$, define the spherical density $\rho(u)=\sum_{i=1}^N c_i\delta(u-u_i)$. Let $Y_{\ell m}$ be orthonormal spherical harmonics and define coefficients $a_{\ell m}=\sum_i c_iY_{\ell m}^*(u_i)$. The angular power at degree $\ell$ is $P_\ell=(2\ell+1)^{-1}\sum_{m=-\ell}^{\ell}|a_{\ell m}|^2$, with monopole $a_{00}=N/\sqrt{4\pi}$. Given a target spectrum $P_\ell^\star$, optimize the neural loss plus $\lambda\sum_{\ell=1}^{L}w_\ell(P_\ell-P_\ell^\star)^2$. The key mechanism is that low $P_\ell$ suppresses large-scale angular density fluctuations, while a positive target at one degree $\ell_0$ creates a controlled spectral feature. Gradients are obtained by differentiating $Y_{\ell m}(u_i)$ through the normalized embedding $u_i=z_i/(\|z_i\|+\epsilon)$.

## Key formulas

- $$\rho(u)=\sum_{i=1}^{N}c_i\,\delta(u-u_i),\qquad u_i\in\mathbb S^2,\qquad \sum_{i=1}^{N}c_i=N.$$
- $$a_{\ell m}=\int_{\mathbb S^2}\rho(u)Y_{\ell m}^{*}(u)\,d\Omega=\sum_{i=1}^{N}c_iY_{\ell m}^{*}(u_i),\qquad P_\ell=\frac{1}{2\ell+1}\sum_{m=-\ell}^{\ell}|a_{\ell m}|^2.$$
- $$\mathcal L_{\mathrm{spec}}=\lambda\sum_{\ell=1}^{L}w_\ell\left(P_\ell-P_\ell^{\star}\right)^2,\qquad \mathcal L=\mathcal L_{\mathrm{task}}+\mathcal L_{\mathrm{spec}}+\mu\mathcal L_{\mathrm{rep}}.$$
- $$\mathcal L_{\mathrm{rep}}=\frac{1}{N(N-1)}\sum_{i\neq j}\left[\max\left(0,\,\cos^{-1}(u_i^{\mathsf T}u_j)-\alpha\right)\right]^2.$$

## Implementation notes

Integrate this at the representation layer immediately after producing a batch of vectors $z_i\in\mathbb R^3$ or after projecting arbitrary embeddings to three dimensions with a learned linear map. Normalize with $u_i=z_i/(\|z_i\|+\epsilon)$, convert each $u_i$ to $(\theta_i,\phi_i)$, and evaluate real or complex spherical harmonics up to degree $L$ using a differentiable library. Pseudocode is: `u = normalize(z); a[l,m] = sum_i(c[i] * conj(Y(l,m,u[i]))); P[l] = mean_m(abs(a[l,m])**2); Lspec = lam * sum_l(w[l]*(P[l]-Ptarget[l])**2); loss = task_loss + Lspec + mu*repulsion(u); backprop(loss)`. For an MVP, use $L=8$, equal weights $c_i=1$, and compute the harmonics directly; later replace the dense evaluation by a nonuniform spherical-harmonic transform when $N$ and $L$ are large. The paper supplies the density-to-harmonic construction and the target-spectrum optimization principle; the neural-network gradient and batching are estimated by automatic differentiation. First test a 3-layer MLP on CIFAR-10 or a contrastive image encoder on CIFAR-10 with a 3D hyperspherical prototype layer. Compare no regularizer, cosine repulsion, and spectral regularization with $P_\ell^\star=0$ for $1\leq\ell\leq4$. The quantitative prediction is that the measured batch spectrum satisfies $P_\ell/P_0$ substantially below the unregularized baseline for low degrees and approaches the target with error decreasing as optimization proceeds. Sweep $\lambda$ and predict a stability boundary: when the regularizer gradient norm becomes comparable to the task gradient, approximately $\lambda\|\nabla\mathcal L_{\mathrm{spec}}\|/\|\nabla\mathcal L_{\mathrm{task}}\|\approx1$, training should begin to oscillate or classification loss should increase. Also measure angular-count variance in spherical caps; low-mode suppression should reduce large-cap variance relative to random embeddings.

## Verification

- Status: mechanism_failed
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built a differentiable spherical-harmonic spectrum regularizer using the Legendre addition theorem, with autodiff and optimization sweeps. The core math passed: the addition-theorem l=1 error was 1.1e-16 and the finite-difference/autodiff gradient relative error was 1.35e-8. The mechanism produced a real monotonic signal: q1 fell from 1.000 at lambda=0 to 0.391 at lambda=1, while cap-count variance fell from 432.5 to 73.1. However, the strict stage-1 predictions were not all confirmed: the observed transition began near lambda=0.03, while the initial gradient-ratio boundary was near lambda=0.1-1, and the lambda=0.1 convergence sweep moved spectrum loss from 0.974 to 1.031 rather than decreasing toward the zero-spectrum target; therefore worked=false.

### Mechanism check

- Verdict: Built a differentiable spherical-harmonic spectrum regularizer using the Legendre addition theorem, with autodiff and optimization sweeps. The core math passed: the addition-theorem l=1 error was 1.1e-16 and the finite-difference/autodiff gradient relative error was 1.35e-8. The mechanism produced a real monotonic signal: q1 fell from 1.000 at lambda=0 to 0.391 at lambda=1, while cap-count variance fell from 432.5 to 73.1. However, the strict stage-1 predictions were not all confirmed: the observed transition began near lambda=0.03, while the initial gradient-ratio boundary was near lambda=0.1-1, and the lambda=0.1 convergence sweep moved spectrum loss from 0.974 to 1.031 rather than decreasing toward the zero-spectrum target; therefore worked=false.
- Confidence: 8/10
- Limitations: Only a synthetic north-pole-collapse toy task was tested; no CIFAR-10, neural representation learning, pairwise-repulsion baseline, learned projection, weighted points, selected nonzero target spectrum, GPU benchmark, or FLOP/runtime comparison was performed.

## Artifacts

- [report.md](https://synthcore.org/code/1079/report.md)
- [results.json](https://synthcore.org/code/1079/results.json)
- [spectrum_experiment.py](https://synthcore.org/code/1079/spectrum_experiment.py)
- [Download all files as ZIP](https://synthcore.org/download/1079)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.