Bounded Exponential Anchor Memory
Implementation & benchmark of arXiv:2608.25298 — Sequential Euclidean tree construction with exponential memory: distributional performance and worst-case guarantees
Source paper: Sequential Euclidean tree construction with exponential memory: distributional performance and worst-case guarantees arXiv:2608.25298 ⓘ · analyzed Aug 29, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace a growing stream of cached token or event representations with one normalized exponentially updated anchor. Use the anchor as a global context vector or as a compressed key/value memory; the state retains recent information while avoiding the full movement cost of repeatedly attaching to the previous token.
Formulas
Mathematical statement
The paper defines points p_0,p_1,...,p_N in the Euclidean unit ball B_2^d(1), and the state recursion x_0=p_0 and x_i=gamma x_{i-1}+(1-gamma)p_i, where gamma is in [0,1]. Expanding the recursion gives x_i=gamma^i p_0+(1-gamma) sum_{j=1}^i gamma^{i-j}p_j, so x_i is a convex combination of observed points and remains in B_2^d(1). The insertion cost is C_N=sum_{i=1}^N ||p_i-x_{i-1}||^alpha, where alpha>0. For arbitrary input sequences and 0<alpha<=3, the paper states that the largest asymptotic mean cost is (2/(1+gamma))^alpha, strictly below the path value 2^alpha when gamma>0. For independent uniform points, the minimizing constant parameter obeys 1-gamma_N^*=Theta(N^{-1/2}); for alpha=2, gamma_N^*=1-N^{-1/2}+1/(2N)+O(N^{-3/2}). In the neural adaptation, p_i is a normalized token embedding and x_i is the compressed memory vector.
Implementation notes
Integrate the method into a streaming Transformer at the layer that produces per-token representations. Project each representation h_t into a memory dimension d and normalize it as p_t=h_t/(||h_t||_2+epsilon), so the unit-ball assumption is approximately satisfied. Maintain one state vector x_t per sequence, layer, or attention head. Before processing token t, use x_{t-1} as a global context token, or create one compressed key/value pair k_t=W_k x_{t-1} and v_t=W_v x_{t-1}. After the token is processed, update the state with x_t=gamma x_{t-1}+(1-gamma)p_t. Pseudocode is: initialize x=p_0; for each token, compute p=normalize(project(h)); compute context=f(x); compute the network output using h and context; then set x=gamma*x+(1-gamma)*p. For a context horizon N, initialize gamma=1-1/sqrt(N)+0.5/N using the paper's alpha=2 expansion, and compare this with gamma values 0.9, 0.99, and 0.999. The exact mathematical computations are the recurrence, geometric weights, and worst-case proxy B(gamma,alpha)=(2/(1+gamma))^alpha. Estimate empirically the embedding radius, average attachment cost, perplexity, and gradient variance. Start with a small causal Transformer on WikiText-2 or enwik8, comparing full KV caching, last-token memory, mean pooling, and this one-vector memory. Measure peak KV memory, tokens per second, validation perplexity versus context length, and worst-decile attachment cost. Success requires a large memory reduction with less perplexity degradation than last-token compression at equal parameter count.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.