RPA Phase-Separation Regularizer

First, integrate the regularizer after the network produces a batch of embeddings or MoE routing probabilities. Partition the batch into $K$ components using known labels, modality IDs, augmentation groups, or soft expert assignments; in an MoE, set $\phi_i$ to the batch mean router probability for expert $i$. Give each component a descriptor $\mathcal T_i$ from a learned embedding table and compute $r_i=\operatorname{softplus}(w^\top\mathcal T_i)+\epsilon$. Precompute a fixed positive quadrature grid $k_q$ and weights $u_q$ on $[k_{\min},k_{\max}]$. For each training batch, compute $g_{iq}=\exp[-(k_qr_i)^2]$, $S_q=\sum_i\phi_i\sigma_i^2g_{iq}$, and $A_q=4\pi l_B/(k_q^2b^3)$. Evaluate the correlation term as $b^3/(4\pi^2)\sum_q u_qk_q^2[\log(1+A_qS_q)-A_q\sum_i\phi_i\sigma_i^2]$, then add $\sum_i(\phi_i/N_i)\log(\phi_i+\epsilon)$ and $0.5\sum_{ij}\chi_{ij}\phi_i\phi_j$. Add $\lambda_{\mathrm{RPA}}f$ to the task loss and backpropagate through routing fractions and topology descriptors. The paper supplies the log-response structure; the Gaussian form factor, cutoffs, quadrature resolution, and coefficients are engineering choices. Use $k_{\min}>0$ for numerical safety and clamp the logarithm argument away from zero. Start with fixed $N_i$, $\sigma_i$, and $\chi_{ij}$; only learn $r_i$ after verifying the fixed-form-factor baseline. On CIFAR-10, train a small four-expert MoE and compare standard load balancing with the regularizer. Sweep $\lambda_{\mathrm{RPA}}$ and $l_B$, measuring accuracy, expert entropy, class-expert mutual information, load variance, and dead-expert frequency. Success for specialization is improved accuracy or class-expert mutual information at equal FLOPs without unacceptable load imbalance; success for invariance is improved corruption or augmentation robustness when the sign or component assignment is chosen to discourage separation.