Superperiodic Frieze Sequence Mixer

Integrate the recurrence as a drop-in token mixer in a pre-norm Transformer block, initially replacing self-attention while retaining the MLP and residual path. Given H of shape B by T by d, project x_i = W_h h_i and generate coefficients with a small gate network: alpha = softplus(W_alpha LayerNorm(H)), producing alpha_i^j for j from 1 to a-1. Initialize alpha_i^1 near 1 and the remaining coefficients near zero. Maintain the previous a states and compute each state with the exact recurrence: v_i equals the sum over j of (-1)^(j-1) alpha_i^j v_{i-j}, plus (-1)^(a-1) v_{i-a}. Apply an output projection and add the result through the residual connection. For fixed-length training chunks of length b, enforce the paper's boundary by initializing the recurrence from the final a states of the preceding chunk with v_{i+b} = (-1)^(a-1) v_i; if strict circularity harms causal modeling, use overlap-and-crop instead. The exact computation from the paper is the recurrence and periodic coefficient tying. Tune a in {2,4,8,16}, and compare scalar gates, per-channel diagonal gates, and input-independent periodic coefficients. Construct the block companion matrix for the average coefficients and monitor its largest eigenvalue as a stability diagnostic. First compare against a 6-layer causal Transformer and a diagonal state-space model of equal hidden size on WikiText-2 plus Copy and Adding synthetic tasks. Measure perplexity or sequence accuracy at equal FLOPs, wall-clock throughput, and gradient-norm variance. Success is at least 1.5 times higher token throughput than dense attention for sequence lengths of 512 or more, with no more than 5 percent perplexity degradation; on long-range synthetic tasks, the periodic closure should improve loss stability over an unconstrained learned recurrence.