Limits of Stochastic Semigroups and Block-Triangular Majorisation
arXiv:2609.04057
2026
Architecture
1 ideas extracted · analyzed Sep 4, 2026
What the math gives to ML
The paper gives a constructive description of stochastic maps that preserve a Gibbs distribution when inverse temperature tends to infinity. Its key transferable asset is that this limit is not obtained by simply substituting the limiting distribution into the finite-temperature constraint: it produces block-upper-triangular stochastic operators with substochastic cross-block flow and a finite extremal decomposition. This suggests an energy- or hierarchy-aware neural mixing layer whose transition matrix preserves a prescribed Gibbs-like distribution while annealing toward one-way information flow between learned groups. The most practical first test is a constrained attention or MoE router, compared with ordinary softmax routing at equal compute.
Ideas from this paper
Unverified
2026
Replace an unconstrained attention or MoE routing matrix with a row-stochastic matrix that preserves a learned Gibbs-like distribution over groups. Anneal its temperature so that the router converges to a block-upper-triangular operator: mixing remains flexible within equal-energy groups, while cross-group traffic becomes directional rather than oscillating or collapsing. This creates a hierarchy-aware inductive bias and can reduce the number of active expert or attention connections.
Useful6/10
Difficulty6/10
Novelty7/10