Reciprocal Beta-Angle Mixer
Source paper: Universal Beta Incidence Angles: Cauchy Rigidity and Infinite Arrangements arXiv:2609.00603 ⓘ · analyzed Sep 2, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Insert a projection-space mixer that combines several fixed or learned directions using reciprocal correlations with the current feature, then normalize the result. The exact construction has a universal beta law for its squared input-output cosine, so it can create controlled angular diversity while remaining deterministic and independent of the chosen direction dictionary.
Formulas
Mathematical statement
For nonzero vectors a_1,...,a_k in R^p, nonnegative weights w_1,...,w_k with sum_j w_j=1, and U uniform on the sphere S^{p-1}, define g_{a,w}(U)=sum_j w_j a_j/(a_j^T U) away from the hyperplanes a_j^T U=0. Since U^T g_{a,w}(U)=sum_j w_j=1, its normalized output N_{a,w}(U)=g_{a,w}(U)/||g_{a,w}(U)|| has squared incidence cosine {U^T N_{a,w}(U)}^2=1/||g_{a,w}(U)||^2. The paper's central identity states that this random variable is exactly Beta(1/2,(p-1)/2), independently of k, the directions, their rank, and the simplex weights. In a neural layer, x/||x|| plays U, the a_j are learned or fixed unit prototype directions, and the beta identity motivates preserving the reciprocal mixer. The singular reciprocal must be replaced in code by a clipped reciprocal, so the implemented layer is an approximation to the exact theorem.
Implementation notes
1. Integration point: apply the mixer to a normalized embedding or transformer residual vector x in R^p, preferably in a projection head or after an MLP rather than directly inside every attention block. Maintain k prototype vectors A in R^{k by p}; initialize them as random unit vectors, and parameterize weights with w=softmax(theta) so they remain nonnegative and sum to one. 2. Pseudocode: compute u=x/(norm(x)+delta); t=A@u; t_clip=sign(t)*clamp(abs(t),min=eps); coefficients=w/t_clip; g=sum_j coefficients[j]*A[j]; y=norm(x)*g/(norm(g)+delta); return residual_scale*y+(1-residual_scale)*x. 3. The paper supplies the exact simplex reciprocal construction and target law. Empirically measure the fraction of clipped projections, the distribution of q=(u dot normalize(g))^2, and its Wasserstein or Kolmogorov distance to Beta(1/2,(p-1)/2). Sweep eps in {1e-2,1e-3,1e-4} and optionally stop prototype gradients when clipping exceeds 5 percent. 4. First experiment: use a 2-layer MLP on CIFAR-10 or a small ViT with a 128-dimensional penultimate embedding. Compare the reciprocal mixer against an ordinary learned linear projection, a normalized random-prototype mixer, and an identity residual at equal parameter count. Test accuracy, calibration, representation uniformity, gradient-norm variance, and throughput. A positive result is improved test accuracy or linear-probe performance at unchanged FLOPs, with q close to the predicted beta law and no more than 10 percent throughput loss.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.