Random Reshuffling Dominates Stochastic Gradient Descent

What the math gives to ML

The paper provides a theoretical justification for replacing with-replacement stochastic sampling by random reshuffling on finite-sum objectives. Its transferable asset is the claimed finite-horizon dominance of random reshuffling for smooth convex component losses, even when the stepsize is not restricted by the traditional threshold proportional to 1/n. The direct neural-network adaptation is an optimizer and dataloader policy: generate one fresh permutation per epoch and ensure that every example is consumed exactly once. The extracted material does not contain the quantitative dominance theorem, so implementation should test the qualitative claim and compare convergence at equal examples processed and equal wall-clock time.