Entropy-Budgeted Contiguous KV Pooling

Integrate this at transformer inference between attention-score computation and KV-cache insertion. For a query or a small group of queries, compute attention probabilities p over the n context positions. Choose a target number of retained groups m and an entropy ceiling R. Each group is a contiguous interval; replace its keys and values by K'_k=(sum_{i in group k} p_i K_i)/(q_k+epsilon) and V'_k=(sum_{i in group k} p_i V_i)/(q_k+epsilon), while storing q_k and the interval boundaries. For an exact small-model prototype, enumerate all boundary tuples b_1,...,b_{m-1}, compute q and H(q), discard partitions with H(q)>R, and select the feasible partition with largest H(q). This costs O(C(n-1,m-1)nm) and is suitable for n<=32. For the scalable version, initialize singleton blocks and perform n-m adjacent merges. At each step calculate Delta for every neighboring pair, tentatively apply each candidate, and retain the merge producing the largest final entropy among candidates satisfying H<=R; use a two-step lookahead when the immediate merge overshoots the budget. Cache partitions at chunk boundaries or compute them from averaged attention over heads to amortize overhead. The formulas above are computed exactly; the only empirical choices are m, R, whether p is per-head or averaged, and whether K/V pooling uses attention weights or uniform weights. First test on a 1-2B decoder on WikiText-103 or C4 with context length 2048, comparing full KV attention, uniform contiguous pooling, and a standard heuristic such as importance-based eviction. Sweep m in {256,512,1024} and R in {log2(m)/2, log2(m)*0.75, log2(m)}. Measure perplexity at equal retained-cache size, attention-output cosine error, decode throughput, and peak KV memory. Success is at least 1.5x decode speed or 2x lower KV memory at a fixed perplexity degradation, with the entropy-bound diagnostic correlating with the observed error.