Augment a causal sequence model with a small hierarchy of prefix summaries weighted by powers of the logarithmic rank of each preceding token. The summaries retain order-sensitive deviations from a baseline representation while costing O(KNd) for sequence length N, hierarchy width K, and hidden dimension d, instead of O(N^2d) dense attention.
Replace the linear state transition in a small recurrent or state-space module by a circulant matrix acting on a vector over a finite field. The hidden state then has only finitely many possible values and follows an exactly periodic orbit after at most \(q^n\) states, eliminating numerical drift on modular-counting and symbolic-memory tasks. A learned real-valued encoder and decoder can surround the discrete core, while the transition itself is fixed, searched, or trained with a…
Replace a dense channel-mixing matrix in a sequence layer with alternating diagonal propagation and sparse unipotent Stokes jumps. The diagonal part carries independently controlled exponential phases, while the unipotent factors implement cheap residual-like mode conversion without changing determinant or requiring a dense matrix multiply. Constrain the phase magnitudes and jump coefficients during training to obtain a reversible, norm-monitorable mixer.
Replace an eigendecomposition-based spectral controller in a small recurrent or state-space transition layer with explicit polynomial projectors. Each hidden state is split into invariant modes, and each mode receives a separately constrained recurrent multiplier, enabling direct suppression of unstable modes or selective retention of long-memory modes using only matrix-polynomial evaluations.