Random walks in Dirichlet random environment in dimension $d+1$
arXiv:2607.20279
2026
Regularization
1 ideas extracted · analyzed Aug 30, 2026
What the math gives to ML
The paper provides a transferable replica-overlap mechanism: Dirichlet-distributed transition probabilities create an exactly computable enhancement of correlated path pairs, and the second moment of point-to-point probabilities is governed by the collision local time of two replicas. In the random-walk model, this produces weak- versus strong-disorder behavior, heavy-tailed endpoint probabilities, and a phase transition in dimension three. A neural analogue is to use Dirichlet stochastic routing or stochastic depth and explicitly control the two-replica overlap of routing paths. This yields a measurable noise and concentration diagnostic rather than an ad hoc regularizer: increasing concentration should suppress overlap variance according to the exact Dirichlet moment ratio, while excessive overlap should predict unstable or poorly diversified computation.
Ideas from this paper
Unverified
2026
Replace deterministic or softmax-only mixture-of-experts routing with a Dirichlet-distributed routing vector and train two independently sampled routing replicas for each token. Penalize excessive replica collision, or adapt the Dirichlet concentration so that routing diversity remains in a prescribed regime. The mechanism comes from the random-environment result that the second moment of a path probability is controlled by the collision local time of two independent replicas.
Useful6/10
Difficulty5/10
Novelty7/10