# Spherical Hemisphere Balance Regularizer

- ID: 1039
- Canonical URL: https://synthcore.org/idea/1039/spherical-hemisphere-balance-regularizer
- API JSON: https://synthcore.org/api/idea/1039.json
- API Markdown: https://synthcore.org/api/idea/1039.md
- Verification status: unverified
- Source: [arXiv:2607.16924](https://arxiv.org/abs/2607.16924)
- Category: regularization
- Solves: generalization, stability
- ML areas: embedding, regularization, contrastive-learning
- Math tags: geometry, metric-geometry, measure-theory, convex-analysis
- Ratings: usefulness 5/10; difficulty 3/10; novelty 7/10

## Idea description

Add a coordinate-free regularizer that prevents a batch of unit-normalized embeddings from concentrating almost entirely on one side of a hyperplane passing through their spherical centroid. Sample random directions tangent to the estimated centroid, measure the soft fraction of embeddings in each corresponding hemisphere, and penalize fractions below the spherical Grünbaum constant. This targets directional mode collapse while preserving rotational invariance.

## Mathematical statement

The spherical Grünbaum inequality states that if K is a spherical convex body in S^{n-1} with centroid theta in S^{n-1}, then for every unit vector u satisfying <u,theta>=0, sigma(K cap u^+) >= c_n sigma(K), where u^+={x in S^{n-1}:<x,u>>=0}, sigma is spherical measure, and c_n=(1-1/n)^{n-1}; the constant is optimal. For embeddings z_i in S^{d-1}, estimate the centroid direction by theta_hat=bar z/||bar z|| with bar z=B^{-1} sum_i z_i. For a random tangent direction u, use the soft hemisphere mass p_+(u)=B^{-1} sum_i sigmoid(<z_i,u>/tau), where tau>0 is a temperature, and p_-(u)=1-p_+(u). The adapted penalty is L_hem=E_u([[c_d-p_+(u)]_+]^2+[[c_d-p_-(u)]_+]^2), with c_d=(1-1/d)^{d-1}. The exact theorem justifies the threshold for uniform spherical-convex mass; the neural loss is an empirical relaxation for finite, nonuniform embedding clouds.

## Key formulas

- $$\sigma(K\cap u^{+})\geq\left(1-\frac{1}{n}\right)^{n-1}\sigma(K),\qquad u\in\mathbb S^{n-1},\quad \langle u,\theta\rangle=0.$$
- $$u^{+}=\{x\in\mathbb S^{n-1}:\langle x,u\rangle\geq0\},\qquad c_n=\left(1-\frac1n\right)^{n-1}.$$
- $$\hat\theta=\frac{\bar z}{\|\bar z\|},\quad \bar z=\frac1B\sum_{i=1}^Bz_i,\quad u=\frac{v-\langle v,\hat\theta\rangle\hat\theta}{\|v-\langle v,\hat\theta\rangle\hat\theta\|},$$
- $$L_{\mathrm{hem}}=\frac1M\sum_{j=1}^M\left(\left[c_d-p_+(u_j)\right]_+^2+\left[c_d-(1-p_+(u_j))\right]_+^2\right),\quad p_+(u_j)=\frac1B\sum_{i=1}^B\operatorname{sigmoid}\!\left(\frac{\langle z_i,u_j\rangle}{\tau}\right).$$

## Implementation notes

(1) Integration point: apply this loss to the output of a projection head, preferably the unit-normalized representation used by a contrastive-learning objective, a hyperspherical VAE prior, or a class-prototype embedding layer. Do not alter the main logits initially; add lambda_hem times L_hem to the ordinary task or contrastive loss during training. Use embedding dimension d and batch size B.

(2) Pseudocode:
```text
z = normalize(projector(h), dim=-1)                 # [B,d]
theta = normalize(stopgrad(mean(z, dim=0)), dim=0)
Lhem = 0
for j in 1..M:
    v = randn(d)
    u = normalize(v - dot(v,theta)*theta)
    s = sigmoid((z @ u) / tau)
    p = mean(s)
    Lhem += relu(c_d-p)^2 + relu(c_d-(1-p))^2
loss = task_loss + lambda_hem * Lhem/M
backprop(loss)
```
Use stop-gradient on theta during the first experiment so the center estimate cannot move to evade the penalty. For noisy batches, maintain theta with an exponential moving average. Set tau between 0.05 and 0.2 and optionally anneal it downward; sample M=8 to 32 tangent directions per batch. Skip the loss when ||bar z|| is below 10^{-3}, because the centroid direction is then undefined.

(3) Computed from the paper: the threshold c_d=(1-1/d)^{d-1} and the requirement that sampled directions be orthogonal to the centroid. Estimated empirically: theta_hat, soft hemisphere masses, and the expectation over random tangent directions. The theorem is an exact certificate only when the support resembles a spherical convex body with approximately uniform measure, so this condition should be checked rather than claimed for arbitrary neural distributions.

(4) First cheap experiment: train a ResNet-18 on CIFAR-10 with a normalized 128-dimensional projection head using SimCLR or supervised contrastive learning. Compare the baseline, standard covariance/variance regularization, and baseline plus this loss at matched FLOPs. Measure linear-probe accuracy, nearest-neighbor accuracy, covariance eigenvalues, worst-direction hemisphere mass over 1,000 random tangent directions, and representation collapse under smaller batches. A successful result is reduced worst-direction imbalance with equal or improved probe accuracy and no increase in training instability. Sweep lambda_hem and compare the theoretically motivated c_d against arbitrary thresholds such as 0.2 and 0.3.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.