Capacity-aware compressive-plus-indexed memory

Integrate this at the output interface of a small SSM, gated recurrent model, or linear-attention backbone. At token $t$, maintain a fixed-width state $\Sigma_t$ and update it with the backbone recurrence; store keys and values $(m_i,v_i)$ for a bounded or external memory. Compute a query key from $q_t$ and retrieve the top $k$ entries using an approximate-nearest-neighbor index, or use dense attention during the first prototype. Form a cheap confidence feature $c_t$, such as the entropy of the state-only output distribution, the margin between its top two logits, or the norm of a state/query compatibility vector. Compute $g_t=\operatorname{sigmoid}(w^\top[\Sigma_t,q_t,c_t])$. If $g_t$ is below a threshold, return the state-only prediction; otherwise attend over the retrieved values and combine them with the state through a two-layer readout. During training, use teacher-forced next-token loss plus a retrieval-budget penalty $\lambda\sum_t g_t$, and optionally force retrieval on examples whose label requires a known distant token. Pseudocode: state=f(state,x[t]); confidence=conf(state,q[t]); gate=sigmoid(linear(state,q[t],confidence)); if gate>tau: R=TopK(index,query(q[t]),k); h=readout(q[t],state,R); else: h=readout(q[t],state); loss+=CE(h,y[t])+lambda*gate. The paper supplies the information-capacity rationale; estimate $B_s$ empirically from state quantization entropy and estimate retrieval cost from measured ANN latency. First test on a synthetic Newton-apple-style stream or copy/retrieval benchmark with sequences of 4k-64k tokens, comparing an equal-width SSM, full attention, and the hybrid. Success means exact distant-retrieval accuracy approaching full attention, state-only failure increasing with context length, and at least 2-4x lower average retrieval/FLOP cost than always-on attention at matched accuracy.