# Reciprocal Beta-Angle Mixer

- ID: 2980
- Canonical URL: https://synthcore.org/idea/2980/reciprocal-beta-angle-mixer
- API JSON: https://synthcore.org/api/idea/2980.json
- API Markdown: https://synthcore.org/api/idea/2980.md
- Verification status: unverified
- Source: [arXiv:2609.00603](https://arxiv.org/abs/2609.00603)
- Category: architecture
- Solves: accuracy, generalization
- ML areas: embedding, mlp, transformer, regularization
- Math tags: probability, geometry, linear-algebra
- Ratings: usefulness 6/10; difficulty 5/10; novelty 8/10

## Idea description

Insert a projection-space mixer that combines several fixed or learned directions using reciprocal correlations with the current feature, then normalize the result. The exact construction has a universal beta law for its squared input-output cosine, so it can create controlled angular diversity while remaining deterministic and independent of the chosen direction dictionary.

## Mathematical statement

For nonzero vectors a_1,...,a_k in R^p, nonnegative weights w_1,...,w_k with sum_j w_j=1, and U uniform on the sphere S^{p-1}, define g_{a,w}(U)=sum_j w_j a_j/(a_j^T U) away from the hyperplanes a_j^T U=0. Since U^T g_{a,w}(U)=sum_j w_j=1, its normalized output N_{a,w}(U)=g_{a,w}(U)/||g_{a,w}(U)|| has squared incidence cosine {U^T N_{a,w}(U)}^2=1/||g_{a,w}(U)||^2. The paper's central identity states that this random variable is exactly Beta(1/2,(p-1)/2), independently of k, the directions, their rank, and the simplex weights. In a neural layer, x/||x|| plays U, the a_j are learned or fixed unit prototype directions, and the beta identity motivates preserving the reciprocal mixer. The singular reciprocal must be replaced in code by a clipped reciprocal, so the implemented layer is an approximation to the exact theorem.

## Key formulas

- $$g_{a,w}(U)=\sum_{j=1}^{k}w_j\frac{a_j}{a_j^{\top}U},\qquad N_{a,w}(U)=\frac{g_{a,w}(U)}{\|g_{a,w}(U)\|}.$$
- $$U^{\top}g_{a,w}(U)=1,\qquad \{U^{\top}N_{a,w}(U)\}^{2}=\frac{1}{\|g_{a,w}(U)\|^{2}}.$$
- $$\{U^{\top}N_{a,w}(U)\}^{2}\sim\operatorname{Beta}\!\left(\frac12,\frac{p-1}{2}\right).$$
- $$\widetilde g(x)=\sum_{j=1}^{k}w_j a_j r_{\varepsilon}(a_j^{\top}u),\quad u=\frac{x}{\|x\|+\delta},\quad r_{\varepsilon}(t)=\frac{\operatorname{sign}(t)}{\max(|t|,\varepsilon)},\quad y=\|x\|\frac{\widetilde g(x)}{\|\widetilde g(x)\|+\delta}.$$

## Implementation notes

1. Integration point: apply the mixer to a normalized embedding or transformer residual vector x in R^p, preferably in a projection head or after an MLP rather than directly inside every attention block. Maintain k prototype vectors A in R^{k by p}; initialize them as random unit vectors, and parameterize weights with w=softmax(theta) so they remain nonnegative and sum to one. 2. Pseudocode: compute u=x/(norm(x)+delta); t=A@u; t_clip=sign(t)*clamp(abs(t),min=eps); coefficients=w/t_clip; g=sum_j coefficients[j]*A[j]; y=norm(x)*g/(norm(g)+delta); return residual_scale*y+(1-residual_scale)*x. 3. The paper supplies the exact simplex reciprocal construction and target law. Empirically measure the fraction of clipped projections, the distribution of q=(u dot normalize(g))^2, and its Wasserstein or Kolmogorov distance to Beta(1/2,(p-1)/2). Sweep eps in {1e-2,1e-3,1e-4} and optionally stop prototype gradients when clipping exceeds 5 percent. 4. First experiment: use a 2-layer MLP on CIFAR-10 or a small ViT with a 128-dimensional penultimate embedding. Compare the reciprocal mixer against an ordinary learned linear projection, a normalized random-prototype mixer, and an identity residual at equal parameter count. Test accuracy, calibration, representation uniformity, gradient-norm variance, and throughput. A positive result is improved test accuracy or linear-probe performance at unchanged FLOPs, with q close to the predicted beta law and no more than 10 percent throughput loss.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.