Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

Unverified 2026

Finite-Orbit Circulant State Core

Replace the linear state transition in a small recurrent or state-space module by a circulant matrix acting on a vector over a finite field. The hidden state then has only finitely many possible values and follows an exactly periodic orbit after at most \(q^n\) states, eliminating numerical drift on modular-counting and symbolic-memory tasks. A learned real-valued encoder and decoder can surround the discrete core, while the transition itself is fixed, searched, or trained with a…

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Periodicities in the Riordan arrays of polynomials over finite fields arXiv:2607.13442