Limits of Stochastic Semigroups and Block-Triangular Majorisation

arXiv:2609.04057 2026 Architecture 1 ideas extracted · analyzed Sep 4, 2026

What the math gives to ML

The paper gives a constructive description of stochastic maps that preserve a Gibbs distribution when inverse temperature tends to infinity. Its key transferable asset is that this limit is not obtained by simply substituting the limiting distribution into the finite-temperature constraint: it produces block-upper-triangular stochastic operators with substochastic cross-block flow and a finite extremal decomposition. This suggests an energy- or hierarchy-aware neural mixing layer whose transition matrix preserves a prescribed Gibbs-like distribution while annealing toward one-way information flow between learned groups. The most practical first test is a constrained attention or MoE router, compared with ordinary softmax routing at equal compute.

Ideas from this paper

Unverified 2026

Zero-Temperature Block-Triangular Routing

Replace an unconstrained attention or MoE routing matrix with a row-stochastic matrix that preserves a learned Gibbs-like distribution over groups. Anneal its temperature so that the router converges to a block-upper-triangular operator: mixing remains flexible within equal-energy groups, while cross-group traffic becomes directional rather than oscillating or collapsing. This creates a hierarchy-aware inductive bias and can reduce the number of active expert or attention connections.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Limits of Stochastic Semigroups and Block-Triangular Majorisation arXiv:2609.04057