△ Mechanism confirmed, baseline not beaten 2026

Equal-Variation KV Cache

Implementation & benchmark of arXiv:2607.09974 — Optimal history encoding for elastic-plastic hereditary laws: Sharp input and constitutive approximation

Usefulness8/10

Difficulty5/10

Novelty6/10

Memory Memory Accuracy Scalability Attention Kv cache Memory Metric geometry Approximation theory Measure theory

Source paper: Optimal history encoding for elastic-plastic hereditary laws: Sharp input and constitutive approximation arXiv:2607.09974 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Compress a causal sequence by retaining history positions at equal increments of cumulative representation variation instead of at uniform time intervals. Use the resulting N representatives in a decoder that reconstructs piecewise-constant keys and values; the paper's minimax result predicts a worst-case reconstruction error of total variation divided by 2N, independent of where rapid changes occur.

Formulas

$$\|\epsilon\|_{E}=\sqrt{\mathbb{C}\epsilon\cdot\epsilon},\qquad \|\sigma\|_{F}=\sqrt{\mathbb{C}^{-1}\sigma\cdot\sigma}.$$

$$V_{\pi}(t)=\int_{0}^{t}\|\dot{\pi}(s)\|_{E}\,ds,\qquad {\rm Var}(\pi)=V_{\pi}(T).$$

$$V_{\pi}(t_j)=\frac{j}{N}{\rm Var}(\pi),\quad j=0,\ldots,N,\qquad \pi_N(t)=\pi(t_j)\ \text{for }t\in[t_j,t_{j+1}).$$

$$\inf_{\mathcal E_N}\sup_{\pi}\frac{\|\pi-\mathcal D(\mathcal E_N(\pi))\|_{L^\infty}}{{\rm Var}(\pi)}=\frac{1}{2N}.$$

Mathematical statement

For an absolutely continuous history $\pi:[0,T]\to E$, define cumulative variation $V_\pi(t)=\int_0^t\|\dot\pi(s)\|_E\,ds$ and total variation ${\rm Var}(\pi)=V_\pi(T)$. The equal-variation encoder chooses breakpoints $0=t_0<\cdots<t_N=T$ satisfying $V_\pi(t_j)=j\,{\rm Var}(\pi)/N$, using a first-hitting-time convention when $V_\pi$ has flat intervals. Its right-continuous step reconstruction $\pi_N(t)=\pi(t_j)$ for $t\in[t_j,t_{j+1})$ has sharp normalized minimax error $1/(2N)$ under the paper's assumptions. The norm $\|\cdot\|_E$ can be replaced by a learned feature-space norm; for transformer compression, $\pi(t)$ is a projected key-value trajectory and variation is estimated by finite differences.

Implementation notes

Integrate this at the long-context attention memory layer, before the attention kernel consumes the KV cache. For each incoming token t, form a low-dimensional feature z_t=P[h_t;k_t;v_t], where P is a fixed random projection or a small learned projection, and estimate local variation d_t=||z_t-z_{t-1}||_2. Maintain cumulative variation c_t=sum_{r=1}^t d_r. In an offline or chunked implementation, define thresholds q_j=j c_T/N and choose each segment boundary as the first token whose cumulative variation crosses the next threshold. Store a representative key and value for each segment, using variation-weighted averages kbar_j=sum_{r in S_j} d_r k_r/(sum_{r in S_j} d_r) and the analogous value average. Retain a recent uncompressed window to preserve local token identity. In streaming inference, use reservoir splitting: when a segment exceeds its variation budget, split it at the first threshold crossing and merge adjacent low-variation segments when necessary. The paper supplies the equal-variation allocation and 1/N scaling; P, the representative statistic, and the online policy are empirical choices. First test a small Transformer-XL or 1B decoder on PG19 and LongBench against uniform eviction, recent-window caching, and H2O at equal KV memory. Measure perplexity, retrieval accuracy, attention FLOPs, and reconstruction error versus N. Success is lower perplexity at fixed cache size and an empirical error curve close to 1/N, especially on sequences with bursts of rapid feature change.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: No measurable effect

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built an equal-variation KV compression MVP with first-hitting cumulative-variation boundaries, representative averaging, trajectory-error checks, and a synthetic attention comparison. The sharp scalar prediction matched 1/(2N) exactly through N=32; burst-path errors decayed with fitted slope -1.12 and remained below the 1/N bound, while equal-variation location sensitivity was substantially lower than uniform timing. However, attention-output MSE generally favored uniform compression, so the mathematical mechanism manifested but an end-to-end accuracy win was not demonstrated.

Agent confidence: 8/10

Baseline: Uniform-time trajectory error mean: 0.4625, 0.3191, 0.1834, 0.0956 for N=4,8,16,32; synthetic attention MSE was usually lower than equal-variation (for example at burst=0.5,N=16: 8.94e-06).
Idea: Equal-variation trajectory error mean: 0.1926, 0.0872, 0.0396, 0.0187 for N=4,8,16,32; N*error decreased from 0.770 to 0.599, fitted log-log slope -1.12. Scalar monotone-path errors were 0.25, 0.125, 0.0625, 0.03125, 0.015625 versus exact 1/(2N).

Limitations: No real Transformer, language-model perplexity, PG19/LongBench, H2O comparison, learned projection, streaming reservoir policy, or GPU benchmark was tested. The attention experiment is a small synthetic proxy and uses offline segment means rather than a production KV-cache kernel.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.09974) Sep 2, 2026

No significant difference vs. the tuned baseline (+0.5%, p=0.641, wins 3 / 8 paired seeds).

Benchmark: Sequence modeling
Model: transformer_tiny
Paired seeds: 8
Baseline mean: 1.2836
Idea mean: 1.2895
Effect (Δ): 0.0059 (+0.5%; negative = idea better)
Wins: 3 / 8 paired seeds
p-value: 0.641 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±2.4%
Mechanism: Confirmed ✓
Practical effect: No effect
Baseline tuning: swept over 3 configs

Limitations:

Only the registered sequence track was tested with 256 training examples, 128 test examples, and 3 epochs. Production long-context KV-cache kernels, language-model perplexity, recent-window/H2O baselines, learned projections, and streaming reservoir splitting were not tested.

How to run: python3 bench_run.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 30, 2026 latest Failed ✗

Agent confidence: 9/10

Equal-variation compression was implemented and evaluated on the registered sequence track with transformer_tiny, using uniform compression as the matched baseline and the same shared hyperparameter grid. The mechanism signature was quantitatively confirmed, but the idea had higher MSE than baseline and no significant win: delta_mean=+0.005932, p=0.64095.

Baseline: Mean MSE 1.2835911363, std 0.1447072547; best config {lr:0.001,N:8}.
Idea: Mean MSE 1.2895227224, std 0.1283316803; best config {lr:0.001,N:8}; 3/8 paired wins.
Mechanism signature: map[confirmed:true observed:0.055078334848618044 predicted:0.0625 quantity:normalized L_inf input-history reconstruction error measured on trained benchmark inputs relative_error:0.11874664242021135]

Benchmark result