Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

2 ideas found

Category: Architecture Solves: Inference speedup ML: Rnn Clear all

Unverified 2026

Finite-Orbit Circulant State Core

Replace the linear state transition in a small recurrent or state-space module by a circulant matrix acting on a vector over a finite field. The hidden state then has only finitely many possible values and follows an exactly periodic orbit after at most \(q^n\) states, eliminating numerical drift on modular-counting and symbolic-memory tasks. A learned real-valued encoder and decoder can surround the discrete core, while the transition itself is fixed, searched, or trained with a…

Useful6/10

Difficulty6/10

Novelty8/10

Architecture Stability Memory Inference speedup

Paper: Periodicities in the Riordan arrays of polynomials over finite fields arXiv:2607.13442

Unverified 2026

Polynomial Spectral Mode Controller

Replace an eigendecomposition-based spectral controller in a small recurrent or state-space transition layer with explicit polynomial projectors. Each hidden state is split into invariant modes, and each mode receives a separately constrained recurrent multiplier, enabling direct suppression of unstable modes or selective retention of long-memory modes using only matrix-polynomial evaluations.

Useful5/10

Difficulty6/10

Novelty6/10

Architecture Stability Accuracy Inference speedup

Paper: A Direct Polynomial Approach to Spectral Decomposition arXiv:2607.20218