Mechanism works
2026
Modular collision-free sparse attention
Replace dense attention between grid-arranged tokens by a deterministic block-sparse pattern generated from modular permutations. In each block, connect row token i to column token p(i)=2i modulo B; because i, i-p(i), and i+p(i) are injective modulo B when gcd(B,6)=1, the pattern avoids repeated horizontal, vertical, and diagonal projections. Use shifted permutations across heads to increase receptive-field coverage while retaining structured sparsity.
Paper: Closing the gap and settling the problem of queens on an $n\times n$ board, each attacking at most one other
arXiv:2608.27432