Entropy-Smooth Convex Optimization Cannot Be Accelerated

What the math gives to ML

The paper proves that convex objectives relatively smooth with respect to negative Shannon entropy cannot generally obtain an accelerated O(1/T^2) rate: when the simplex dimension satisfies d=Omega(T^2), every first-order method has error at least Omega(L/T). The transferable asset is a principled warning against blindly adding Euclidean or Nesterov momentum to high-dimensional probability-valued neural variables. A practical experiment is to use entropy mirror descent for MoE router probabilities, attention distributions, or mixture weights and compare it with accelerated updates under matched gradient evaluations.