DynaBase Retrieval Forecast Head
Implementation & benchmark of arXiv:2607.14937 — A Minimal Interpretable Architecture for Zero-Shot Reconstruction of Dynamical Systems
Source paper: A Minimal Interpretable Architecture for Zero-Shot Reconstruction of Dynamical Systems arXiv:2607.14937 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace a parameter-heavy recurrent transition, or use this as a fallback, with a two-parameter nearest-neighbor successor blend in latent space. Given a query latent state, retrieve the closest state from an in-context trajectory and combine the query, the retrieved state, and its observed successor; this gives a zero-shot dynamical forecast with almost no trainable transition parameters.
Formulas
Mathematical statement
The paper motivates reducing the DynaMix expert transition $\bm{z}^{j}_{t+1}=\bm{A}_{j}\bm{z}_{t}+\bm{W}_{j}\Phi^{*}(\bm{z}_{t})+\bm{h}_{j}$, where $\bm{z}_{t}\in\mathbb{R}^{M}$ is the latent state, $\bm{A}_{j}\in\mathbb{R}^{M\times M}$ is diagonal, $\bm{W}_{j}\in\mathbb{R}^{M\times M}$ is a recurrent interaction matrix, $\bm{h}_{j}\in\mathbb{R}^{M}$ is a bias, and $\Phi^{*}$ applies identity activations to some coordinates and ReLU to the rest. The adapted minimal map uses an in-context trajectory $\{\bm{c}_{k}\}_{k=1}^{T}$, chooses $i(t)=\arg\min_{1\leq k<T}\|\bm{z}_{t}-\bm{c}_{k}\|_{2}$, and uses the successor pair $(\bm{c}_{i(t)},\bm{c}_{i(t)+1})$. The two scalar coefficients $\alpha,\beta$ control the retrieved state and its successor; $1-\alpha-\beta$ weights the current state. If $\alpha,\beta\geq0$ and $\alpha+\beta\leq1$, the prediction is a convex combination and therefore lies in the convex hull of three observed latent states, providing a simple boundedness bias.
Implementation notes
(1) Integration point: attach this head after an encoder $E$ that maps observations $\bm{x}_{t}\in\mathbb{R}^{d}$ to latents $\bm{z}_{t}=E(\bm{x}_{t})\in\mathbb{R}^{M}$. During inference, construct a memory bank from the in-context trajectory $\bm{c}_{k}=E(\bm{x}_{k})$ and their one-step successors. Use the DynaBase transition instead of the recurrent or MLP transition, or ensemble both with a learned gate. Perform nearest-neighbor search over $\bm{c}_{1:T-1}$ using cosine or Euclidean distance; use FAISS for long contexts. (2) Pseudocode: C=encode(context[:-1]); S=encode(context[1:]); z=encode(query); for h in range(H): i=argmin_k distance(z,C[k]); z=(1-a-b)*z+a*C[i]+b*S[i]; prediction=decode(z). Parameterize $(a,b,1-a-b)$ with a three-way softmax so the update is a convex combination. (3) The paper-derived component is the successor retrieval and two-scalar affine map; estimate the encoder, nearest-neighbor index, and coefficients empirically. Initialize logits to favor $\alpha=0.25,\beta=0.5$ and train only the encoder and two logits, optionally fitting logits by least squares on demonstrations before end-to-end fine-tuning. (4) First experiment: compare against a GRU, linear autoregression, and nearest-neighbor successor copying on Lorenz-63, Mackey-Glass, and periodic oscillator trajectories. Use 8-shot contexts and 32-step rollouts. Measure one-step MSE, long-horizon boundedness, parameters, and CPU latency. Success means comparable or lower reconstruction error than GRU with at least 100x fewer transition parameters and fewer exploding rollouts.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.