Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

Unverified 2026

Okada Non-Crossing Routing Attention

Replace dense token-to-token attention with a learned composition of adjacent routing operators whose connectivity is stored as a non-crossing labelled arc diagram. Canonicalize the composed routing program using the Okada relations, then execute only the surviving sparse token paths.

Useful4/10
Difficulty6/10
Novelty7/10
Paper: Diagrammatic Okada monoid and cellularity of the Okada algebra arXiv:2609.01440