Spherical harmonic spectrum regularizer
Implementation & benchmark of arXiv:2608.24867 — Fast generation of spectrally-shaped disorder, on the sphere
Source paper: Fast generation of spectrally-shaped disorder, on the sphere arXiv:2608.24867 ⓘ · analyzed Sep 1, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Constrain a set of learnable or batch-produced unit-norm embeddings by matching their spherical-harmonic power spectrum to a target spectrum rather than relying only on pairwise Euclidean repulsion. This creates an explicit, tunable mechanism for suppressing low-frequency density fluctuations or enhancing a selected angular frequency, which can improve uniformity and reduce representation collapse on hyperspherical embeddings.
Formulas
Mathematical statement
For unit vectors $u_i\in\mathbb S^2$ with optional weights $c_i$ satisfying $\sum_i c_i=N$, define the spherical density $\rho(u)=\sum_{i=1}^N c_i\delta(u-u_i)$. Let $Y_{\ell m}$ be orthonormal spherical harmonics and define coefficients $a_{\ell m}=\sum_i c_iY_{\ell m}^*(u_i)$. The angular power at degree $\ell$ is $P_\ell=(2\ell+1)^{-1}\sum_{m=-\ell}^{\ell}|a_{\ell m}|^2$, with monopole $a_{00}=N/\sqrt{4\pi}$. Given a target spectrum $P_\ell^\star$, optimize the neural loss plus $\lambda\sum_{\ell=1}^{L}w_\ell(P_\ell-P_\ell^\star)^2$. The key mechanism is that low $P_\ell$ suppresses large-scale angular density fluctuations, while a positive target at one degree $\ell_0$ creates a controlled spectral feature. Gradients are obtained by differentiating $Y_{\ell m}(u_i)$ through the normalized embedding $u_i=z_i/(\|z_i\|+\epsilon)$.
Implementation notes
Integrate this at the representation layer immediately after producing a batch of vectors $z_i\in\mathbb R^3$ or after projecting arbitrary embeddings to three dimensions with a learned linear map. Normalize with $u_i=z_i/(\|z_i\|+\epsilon)$, convert each $u_i$ to $(\theta_i,\phi_i)$, and evaluate real or complex spherical harmonics up to degree $L$ using a differentiable library. Pseudocode is: u = normalize(z); a[l,m] = sum_i(c[i] * conj(Y(l,m,u[i]))); P[l] = mean_m(abs(a[l,m])**2); Lspec = lam * sum_l(w[l]*(P[l]-Ptarget[l])**2); loss = task_loss + Lspec + mu*repulsion(u); backprop(loss). For an MVP, use $L=8$, equal weights $c_i=1$, and compute the harmonics directly; later replace the dense evaluation by a nonuniform spherical-harmonic transform when $N$ and $L$ are large. The paper supplies the density-to-harmonic construction and the target-spectrum optimization principle; the neural-network gradient and batching are estimated by automatic differentiation. First test a 3-layer MLP on CIFAR-10 or a contrastive image encoder on CIFAR-10 with a 3D hyperspherical prototype layer. Compare no regularizer, cosine repulsion, and spectral regularization with $P_\ell^\star=0$ for $1\leq\ell\leq4$. The quantitative prediction is that the measured batch spectrum satisfies $P_\ell/P_0$ substantially below the unregularized baseline for low degrees and approaches the target with error decreasing as optimization proceeds. Sweep $\lambda$ and predict a stability boundary: when the regularizer gradient norm becomes comparable to the task gradient, approximately $\lambda\|\nabla\mathcal L_{\mathrm{spec}}\|/\|\nabla\mathcal L_{\mathrm{task}}\|\approx1$, training should begin to oscillate or classification loss should increase. Also measure angular-count variance in spherical caps; low-mode suppression should reduce large-cap variance relative to random embeddings.
Verification
Stage 1 · Toy mechanism gate: Failed ✗
Stage 2 · Mechanism transferred to benchmark: Not tested
Stage 2 · Practical benchmark result: Not run
Methodology: Toy-system gate first; the benchmark stage runs only after a pass. How verification works
Stage 1 — Mechanism check agent confidence 8/10
Built a differentiable spherical-harmonic spectrum regularizer using the Legendre addition theorem, with autodiff and optimization sweeps. The core math passed: the addition-theorem l=1 error was 1.1e-16 and the finite-difference/autodiff gradient relative error was 1.35e-8. The mechanism produced a real monotonic signal: q1 fell from 1.000 at lambda=0 to 0.391 at lambda=1, while cap-count variance fell from 432.5 to 73.1. However, the strict stage-1 predictions were not all confirmed: the observed transition began near lambda=0.03, while the initial gradient-ratio boundary was near lambda=0.1-1, and the lambda=0.1 convergence sweep moved spectrum loss from 0.974 to 1.031 rather than decreasing toward the zero-spectrum target; therefore worked=false.
- Agent confidence
- 8/10
- Baseline
- lambda=0: task=-1.0000, normalized spectrum loss=4.0000, q1=1.0000, cap variance=432.538
- Gradient scaling
- spectrum/task gradient norm ratio=8.565 at lambda=1, 0.857 at lambda=0.1, 0.0857 at lambda=0.01
- Idea
- lambda=1: task=-0.6256, normalized spectrum loss=0.1538, q1=0.3914, cap variance=73.128; lambda=0.1 gives task=-0.9065, spectrum loss=1.0307, q1=0.8217, cap variance=268.560
- Lambda sweep
- q1: 1.000 at lambda=0, 1.000 at .01, .930 at .03, .822 at .1, .666 at .3, .391 at 1
- Math check
- addition-theorem absolute error=1.11e-16; gradient finite-difference relative error=1.35e-8
Limitations: Only a synthetic north-pole-collapse toy task was tested; no CIFAR-10, neural representation learning, pairwise-repulsion baseline, learned projection, weighted points, selected nonzero target spectrum, GPU benchmark, or FLOP/runtime comparison was performed.
How to run: python3 spectrum_experiment.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Artifacts
- report.md 1.9 KB View
- results.json 7.7 KB View Raw
- spectrum_experiment.py 3.1 KB View Raw