# DynaBase Retrieval Forecast Head

- ID: 940
- Canonical URL: https://synthcore.org/idea/940/dynabase-retrieval-forecast-head
- API JSON: https://synthcore.org/api/idea/940.json
- API Markdown: https://synthcore.org/api/idea/940.md
- Verification status: unverified
- Source: [arXiv:2607.14937](https://arxiv.org/abs/2607.14937)
- Category: architecture
- Solves: scalability, stability, accuracy
- ML areas: world-model, retrieval, ssm, inference-speedup
- Math tags: dynamical-systems, metric-geometry, linear-algebra, approximation-theory
- Ratings: usefulness 6/10; difficulty 4/10; novelty 5/10

## Idea description

Replace a parameter-heavy recurrent transition, or use this as a fallback, with a two-parameter nearest-neighbor successor blend in latent space. Given a query latent state, retrieve the closest state from an in-context trajectory and combine the query, the retrieved state, and its observed successor; this gives a zero-shot dynamical forecast with almost no trainable transition parameters.

## Mathematical statement

The paper motivates reducing the DynaMix expert transition $\bm{z}^{j}_{t+1}=\bm{A}_{j}\bm{z}_{t}+\bm{W}_{j}\Phi^{*}(\bm{z}_{t})+\bm{h}_{j}$, where $\bm{z}_{t}\in\mathbb{R}^{M}$ is the latent state, $\bm{A}_{j}\in\mathbb{R}^{M\times M}$ is diagonal, $\bm{W}_{j}\in\mathbb{R}^{M\times M}$ is a recurrent interaction matrix, $\bm{h}_{j}\in\mathbb{R}^{M}$ is a bias, and $\Phi^{*}$ applies identity activations to some coordinates and ReLU to the rest. The adapted minimal map uses an in-context trajectory $\{\bm{c}_{k}\}_{k=1}^{T}$, chooses $i(t)=\arg\min_{1\leq k<T}\|\bm{z}_{t}-\bm{c}_{k}\|_{2}$, and uses the successor pair $(\bm{c}_{i(t)},\bm{c}_{i(t)+1})$. The two scalar coefficients $\alpha,\beta$ control the retrieved state and its successor; $1-\alpha-\beta$ weights the current state. If $\alpha,\beta\geq0$ and $\alpha+\beta\leq1$, the prediction is a convex combination and therefore lies in the convex hull of three observed latent states, providing a simple boundedness bias.

## Key formulas

- $$\bm{z}^{j}_{t+1}=F^{\mathrm{exp}}_{j}(\bm{z}_{t})=\bm{A}_{j}\bm{z}_{t}+\bm{W}_{j}\Phi^{*}(\bm{z}_{t})+\bm{h}_{j}.$$
- $$i(t)=\arg\min_{1\leq k<T}\|\bm{z}_{t}-\bm{c}_{k}\|_{2},\qquad \hat{\bm{z}}_{t+1}=(1-\alpha-\beta)\bm{z}_{t}+\alpha\bm{c}_{i(t)}+\beta\bm{c}_{i(t)+1}.$$
- $$\begin{bmatrix}\alpha\\\beta\end{bmatrix}=\operatorname{softmax}_{3}(\bm{q})_{1:2},\qquad 1-\alpha-\beta=\operatorname{softmax}_{3}(\bm{q})_{3},$$
- $$\hat{\bm{z}}_{t+h+1}=\mathcal{D}(\hat{\bm{z}}_{t+h};\,\{\bm{c}_{k}\}_{k=1}^{T}),\quad \mathcal{D}(\bm{x};C)=(1-\alpha-\beta)\bm{x}+\alpha\bm{c}_{i(x)}+\beta\bm{c}_{i(x)+1}.$$

## Implementation notes

(1) Integration point: attach this head after an encoder $E$ that maps observations $\bm{x}_{t}\in\mathbb{R}^{d}$ to latents $\bm{z}_{t}=E(\bm{x}_{t})\in\mathbb{R}^{M}$. During inference, construct a memory bank from the in-context trajectory $\bm{c}_{k}=E(\bm{x}_{k})$ and their one-step successors. Use the DynaBase transition instead of the recurrent or MLP transition, or ensemble both with a learned gate. Perform nearest-neighbor search over $\bm{c}_{1:T-1}$ using cosine or Euclidean distance; use FAISS for long contexts. (2) Pseudocode: `C=encode(context[:-1]); S=encode(context[1:]); z=encode(query); for h in range(H): i=argmin_k distance(z,C[k]); z=(1-a-b)*z+a*C[i]+b*S[i]; prediction=decode(z)`. Parameterize $(a,b,1-a-b)$ with a three-way softmax so the update is a convex combination. (3) The paper-derived component is the successor retrieval and two-scalar affine map; estimate the encoder, nearest-neighbor index, and coefficients empirically. Initialize logits to favor $\alpha=0.25,\beta=0.5$ and train only the encoder and two logits, optionally fitting logits by least squares on demonstrations before end-to-end fine-tuning. (4) First experiment: compare against a GRU, linear autoregression, and nearest-neighbor successor copying on Lorenz-63, Mackey-Glass, and periodic oscillator trajectories. Use 8-shot contexts and 32-step rollouts. Measure one-step MSE, long-horizon boundedness, parameters, and CPU latency. Success means comparable or lower reconstruction error than GRU with at least 100x fewer transition parameters and fewer exploding rollouts.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.