Reversal-Assisted MoE Routing

1. Integration point: use this as a routing layer before each MoE feed-forward block. Represent each expert by a cyclic array of capacity slots; tokens occupy slots and empty slots are vacancies. Initialize each token's direction from the gradient of its router logits toward neighboring experts, or sample directions from the top-two expert logits. Run a small number of routing events before executing experts. 2. Pseudocode: maintain occupancy and direction arrays; at every event compute each token's forward vacancy run length n_i; assign weight w_i=u_infinity(1+b/n_i) when the destination is empty; sample or select a token proportional to w_i; move it one slot; and independently flip every direction with probability 1-exp(-gamma Delta t). Aggregate tokens by expert and use a straight-through or soft relaxation for differentiable training. 3. The paper supplies the hard-core dynamics, inverse-vacancy hop law, and reversal mechanism. Estimate empirically the phase boundary, finite-size corrections, and the mapping from router logits to initial directions. 4. First cheap experiment: compare standard top-1 and top-2 routing against this mechanism in a small Switch-Transformer-style model with 8 experts on CIFAR-100 or synthetic classification. Sweep density rho, reversal rate gamma, and b. Measure mobility M, maximum expert load, dropped-token fraction, cluster length, throughput, and validation loss. The predicted signature is a sharp decrease in M and increase in cluster length near a reproducible rho_c(gamma); decreasing gamma should shift or sharpen the transition, while sufficiently rapid reversal should remove persistent low-mobility jams. Falsify the mechanism if no mobility transition appears or if tail expert load is not reduced at matched capacity.