Fractional-memory recurrent state

Add the module as a drop-in replacement for the temporal mixing block in a small causal Transformer, GRU, or diagonal SSM. For an input sequence tensor \(x\in\mathbb{R}^{T\times B\times d}\), create \(K\) state tensors \(s_k\in\mathbb{R}^{B\times d}\); use \(K=8\), positive weights normalized by softmax, and decay factors \(\rho_k=\exp(-\Delta/\tau_k)\) with logarithmically spaced \(\tau_k\) from 1 to the target context length. At every timestep, compute s[k] = rho[k] * s[k] + (1-rho[k]) * x[t], then y[t] = sum(w[k] * s[k]); apply a learned input/output projection and residual connection around the filter. Optionally learn \(\alpha\) and \(\tau\) only through a differentiable initialization map that refits \(w_k,\rho_k\), but begin with fixed stable decays. Fit the bank before training by minimizing squared error between \(\hat h_\ell\) and the paper kernel \(E_\alpha(-((\ell+1)/\tau)^\alpha)\) over logarithmic lags; this fit is the empirical approximation, while the power-law asymptotic is the mathematical motivation. The first cheap experiment should compare this layer with a GRU, a single-exponential SSM, and a standard causal Transformer on synthetic delayed-copy and adding problems at sequence lengths 1k, 4k, and 16k, followed by TinyStories language modeling. Track accuracy as delay/context grows, validation perplexity at equal parameter count, and memory/runtime. Success is materially better delayed-copy accuracy and less perplexity degradation at long context with \(O(KTd)\) compute and constant recurrent state.