Persistent Relational Memory
Implementation & benchmark of arXiv:2609.02991 — TRACE: Spatiotemporal Contact Memory Graph Network Simulator for Granular Dynamics
Source paper: TRACE: Spatiotemporal Contact Memory Graph Network Simulator for Granular Dynamics arXiv:2609.02991 ⓘ · analyzed Sep 4, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Give every possible pairwise relation its own recurrent latent state, stored in a dictionary keyed by stable node identities, instead of discarding the state whenever the edge is absent from the current graph. At each timestep, active edges retrieve their old state, update it with the current pair features and neighborhood messages, and write it back after message passing. This should improve predictions in dynamic graphs with intermittent interactions and reduce the burden on node states to remember which specific neighbor caused an earlier event.
Formulas
Mathematical statement
The paper constructs the active graph at time t as $\mathcal{G}^{t}=(\mathcal{V},\mathcal{E}^{t})$, with nodes representing particles and $\mathcal{E}^{t}=\{(i,j):\|\mathbf{x}_{i}^{t}-\mathbf{x}_{j}^{t}\|_{2}<\alpha(r_i+r_j),\ i<j\}$. Here $\mathbf{x}_i^t$ is the feature or position of node i at time t, $r_i$ is its radius or entity scale, and $\alpha$ is a skin factor controlling when an edge becomes active. The transferable operation is not the granular radius criterion itself, but the separation between a changing active edge set and a persistent state indexed by the stable pair key $(i,j)$. For each active edge, let $m_{ij}^t$ be its stored memory, $e_{ij}^t$ its current pair feature, and $a_{ij}^t$ its aggregated attention/message input. Use a gated recurrent update $m_{ij}^{t+1}=\operatorname{GRU}(m_{ij}^{t},[e_{ij}^{t};a_{ij}^{t}])$. A concrete GRU expansion is $z=\sigma(W_z u+U_zm+b_z)$, $q=\sigma(W_q u+U_qm+b_q)$, $\tilde m=\tanh(W_m u+U_m(q\odot m)+b_m)$, and $m^+= (1-z)\odot m+z\odot\tilde m$, where $u=[e_{ij}^{t};a_{ij}^{t}]$, $\sigma$ is the sigmoid, and all W,U,b are learned parameters. The dictionary preserves $m_{ij}$ across inactive periods; a new pair receives $m_{ij}=0$ or a learned initialization.
Implementation notes
(1) Integration point: modify a temporal GNN or graph-transformer layer that currently recomputes edge embeddings from the active adjacency at every timestep. Assume each node has a stable integer ID and each timestep provides node features x[t] and a candidate-edge generator. Build the active edge set using the paper's rule when geometric positions and radii exist; otherwise use a task-specific threshold, k-nearest-neighbor graph, or observed event list. Maintain a GPU hash map memory[(min(i,j),max(i,j))] containing a vector of dimension d_mem and optionally a last_seen timestep.
(2) Pseudocode:
memory = empty_map()
for t in range(T):
E = {(i,j): distance(x[i],x[j]) < alpha*(r[i]+r[j])}
for (i,j) in E:
m = memory.get((i,j), learned_zero)
e = edge_encoder(x[i], x[j], edge_attributes[i,j])
a = edge_attention(e, neighboring_active_edges(i,j), node_states)
u = concat(e, a)
z = sigmoid(Wz@u + Uz@m + bz)
q = sigmoid(Wq@u + Uq@m + bq)
candidate = tanh(Wm@u + Um@(q*m) + bm)
m_new = (1-z)*m + z*candidate
memory[(i,j)] = m_new
edge_embedding[i,j] = concat(e, m_new)
node_states = graph_message_passing(node_states, edge_embedding, E)
prediction[t] = decoder(node_states, edge_embedding)
Use detached memory between training chunks only if memory growth causes backpropagation-through-time to exceed the budget; otherwise backpropagate for 8-32 steps. Retain inactive memories for a fixed horizon H, or evict them with an LRU policy after H steps. The attention formula above can be replaced with ordinary local edge aggregation for the first MVP.
(3) Computed from the paper's mathematics: dynamic edge construction, stable pair-key lookup, and the gated recurrence. Estimated empirically: the memory dimension d_mem, retention horizon H, attention neighborhood, and whether inactive memories should decay. Add an optional decay $m\leftarrow\rho^{\Delta t}m$ on retrieval, with learned or tuned $\rho\in(0,1]$, to prevent stale information.
(4) First cheap experiment: use a synthetic 2D multi-agent interaction dataset with 32 particles whose pair interactions switch on when distance is below a threshold, and train a 2-layer message-passing network to predict positions 10-50 steps ahead. Compare (a) no temporal memory, (b) node GRU memory, and (c) persistent edge GRU memory with identical hidden size and FLOPs. Report one-step MSE, 50-step rollout MSE, error immediately after a previously seen edge reappears, and memory/latency overhead. The hypothesis is specifically lower reactivation error and more stable long rollouts for (c), not necessarily lower one-step error. A successful signal is at least 20% lower long-horizon MSE at equal parameter count, with less than 30% inference overhead.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.