Boundary-Safe Log-Barrier Mirror Optimizer

(1) Integrate the method at the optimizer for a simplex-valued tensor, not inside the router network. Let P in R^{B x E} be per-token expert probabilities, with every row positive and summing to one. Compute the minibatch loss gradient G = dL/dP by backpropagation, and maintain P as the constrained trainable state; for a cheap first test, apply the method only to a learned global expert-prior vector. (2) For each simplex row p, initialize alpha = alpha_0, set d_i = p_i^{-1} + alpha(G_i + lambda), and compute q_i = 1/d_i. Find lambda by bisection on F(lambda) = sum_i 1/[p_i^{-1} + alpha(G_i + lambda)] - 1. Restrict the search to values for which every denominator exceeds a numerical floor epsilon_d. If the candidate is nonfinite or causes a large loss increase, halve alpha and retry. Set p to q, optionally enforce p_i >= 10^{-8} for numerical safety, and renormalize. This is the KKT form of the logarithmic-barrier mirror step; do not apply a second Adam update to the same state. (3) The barrier, Bregman divergence, reciprocal update, and simplex root condition come from the paper's mathematics. Estimate gradients by backpropagation, select alpha_0 with a short logarithmic sweep, and perform the reciprocal and bisection calculations in float32 even when the network uses mixed precision. (4) First experiment: train a small Switch-Transformer or top-1 MoE on CIFAR-100 or a toy language-model corpus with 8 experts. Compare AdamW on router logits followed by softmax, exponentiated-gradient routing, and the barrier mirror optimizer at matched optimizer FLOPs. Measure validation loss, expert-load entropy, the fraction of numerically zero routes, gradient NaNs, and loss after a fixed number of updates. Success is lower validation loss or fewer routing collapses at equal training cost, particularly when the useful routing distribution is highly imbalanced. A negative result is consistently slower loss descent or persistent expert under-utilization.