Entropy-Geometry Anti-Acceleration Optimizer

(1) Integrate the method into the optimizer update for a tensor whose final axis is a probability simplex: an MoE router probability vector, attention weights, learned mixture weights, or a categorical policy head. Do not apply it directly to ordinary unconstrained weights. Keep probabilities in float32, compute the loss gradient g_t with respect to the probability tensor x_t, and update each simplex independently. (2) Pseudocode: initialize x = softmax(z); at each minibatch compute loss and g = d loss / d x; compute u_k = log(x_k + epsilon) - eta*g_k; set x_new = softmax(u) over the simplex axis; optionally store z = u. The common additive shift in u is irrelevant. Use eta = 1/L when a bound is available. Otherwise start with eta = 0.1 and perform backtracking: evaluate the proposed step and increase L, equivalently decrease eta, until loss(x_new) <= loss(x) + sum_k g_k*(x_new_k-x_k) + L*sum_k x_new_k*log((x_new_k+epsilon)/(x_k+epsilon)). (3) The mathematical contribution is the entropy Bregman geometry and the Omega(L/T) anti-acceleration lower bound. Estimate L empirically with minibatch backtracking; estimate D_h using the displayed KL expression and epsilon clamping. (4) First experiment: train a 64- or 256-expert Switch-style router on CIFAR-100 or a small language-model MoE. Compare AdamW on router logits, Nesterov-accelerated mirror descent, and entropy mirror descent with equal minibatches, gradient evaluations, and FLOPs. Measure validation loss, loss versus optimizer steps, routing entropy, fraction of dead experts, maximum gradient norm, and expert-load coefficient of variation. The falsifiable prediction is that acceleration will not improve loss scaling at large d and may be less stable, while entropy mirror descent will match or beat it in validation loss and produce fewer dead experts at equal compute.