Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

Unverified 2026

Weighted Tree Dirichlet-to-Neumann Layer

Replace dense long-range token interactions with sparse message passing on a balanced hierarchical tree. Boundary token features are extended to internal tree nodes by weighted harmonic relaxation, and the resulting weighted normal derivative is used as a learned multiscale interaction or regularization signal.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Extension Problems for the Vladimirov--Taibleson Operator and the Hierarchical Laplacian arXiv:2609.03647