Unverified 2026

Prony-Compressed Temporal Memory

Implementation & benchmark of arXiv:2608.11520 — Contour Hankel dynamics and indicator fields for the Riemann $Ξ$-function

Usefulness6/10

Difficulty6/10

Novelty5/10

Memory Memory Speedup Scalability Ssm Kv cache Inference speedup Memory Linear algebra Tensor decomposition Dynamical systems Numerical analysis

Source paper: Contour Hankel dynamics and indicator fields for the Riemann $Ξ$-function arXiv:2608.11520 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Approximate a long hidden-state or key/value trajectory by a small sum of exponential modes, storing only the mode nodes and vector weights instead of every timestep. Recover the modes from a Hankel matrix through an annihilating polynomial, then reconstruct the trajectory or a compact recurrent state during inference.

Formulas

$$y_k=\sum_{j=1}^{p}c_j u_j^k,\qquad H_p=(y_{i+j})_{i,j=0}^{p-1}.$$

$$\begin{pmatrix}y_0&y_1&\cdots&y_{p-1}\\y_1&y_2&\cdots&y_p\\\vdots&\vdots&\ddots&\vdots\\y_{p-1}&y_p&\cdots&y_{2p-2}\end{pmatrix}\begin{pmatrix}a_0\\a_1\\\vdots\\a_{p-1}\end{pmatrix}=-\begin{pmatrix}y_p\\y_{p+1}\\\vdots\\y_{2p-1}\end{pmatrix}.$$

$$\pi(z)=z^p+a_{p-1}z^{p-1}+\cdots+a_0=\prod_{j=1}^{p}(z-u_j),\qquad x_k\approx\sum_{j=1}^{p}v_j u_j^k.$$

$$\begin{pmatrix}1&\cdots&1\\u_1&\cdots&u_p\\\vdots&\ddots&\vdots\\u_1^{p-1}&\cdots&u_p^{p-1}\end{pmatrix}\begin{pmatrix}v_1\\\vdots\\v_p\end{pmatrix}=\begin{pmatrix}x_0\\\vdots\\x_{p-1}\end{pmatrix},\qquad |u_j|\leq 1.$$

Mathematical statement

The paper uses the finite-atomic moment model y_k = sum_{j=1}^p c_j u_j^k, where u_j are distinct nodes, c_j are weights, and p is the number of modes. The Hankel matrix is H_p = (y_{i+j}) for indices i,j from 0 to p-1. Equation (70) solves the annihilating-polynomial coefficients a_0 through a_{p-1}; the polynomial pi(z) = z^p + a_{p-1} z^{p-1} + ... + a_0 has roots u_j. Equation (71) recovers the weights through a Vandermonde system. For a vector trajectory x_k in R^d, use x_k approximately equal to sum_{j=1}^p v_j u_j^k, with vector weights v_j in R^d. The stability restriction |u_j| <= 1 prevents exponentially growing modes and permits recursive inference updates.

Implementation notes

Integrate this as a compressed-memory branch in a recurrent model or as an optional long-context KV-cache compressor. Given a block of hidden states x_0 through x_{L-1} in R^d, choose a mode count p much smaller than L. First form one or several random projections y_k = q^T x_k, using fixed Gaussian vectors q, and build the scalar Hankel matrix H with entries H_{m,n} = y_{m+n}. Estimate the numerical rank using an SVD. For the selected p, solve the annihilating system from the paper with Tikhonov regularization, then compute the roots of pi(z). Reject, damp, or clip roots whose magnitude exceeds 1 plus a small tolerance. With the nodes fixed, solve a vector least-squares problem for v_j using all available x_k rather than only the first p samples. Store only the nodes and vector weights. During inference, reconstruct x_k on demand or update each mode recursively with a_{j,k+1} = u_j a_{j,k}, avoiding storage of all L states. For a transformer, retain a recent exact-token window and apply this compression to older value trajectories; initially compress keys only after projection into a lower-dimensional attention subspace. The paper determines the noiseless recovery equations. The implementation must estimate the SVD threshold, regularization strength, block length, root damping, and refresh frequency empirically. First test a 100M-parameter causal transformer or a small SSM on WikiText-103 or Long Range Arena. Compare full history, uniform pooling, learned low-rank compression, and Prony compression at equal memory. Success is at least a fourfold reduction in historical-state memory with less than one percent validation-loss degradation, plus lower decoding latency or a longer context at fixed GPU memory.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.