Persistent Relational Memory

(1) Integration point: modify a temporal GNN or graph-transformer layer that currently recomputes edge embeddings from the active adjacency at every timestep. Assume each node has a stable integer ID and each timestep provides node features x[t] and a candidate-edge generator. Build the active edge set using the paper's rule when geometric positions and radii exist; otherwise use a task-specific threshold, k-nearest-neighbor graph, or observed event list. Maintain a GPU hash map memory[(min(i,j),max(i,j))] containing a vector of dimension d_mem and optionally a last_seen timestep.

(2) Pseudocode:

memory = empty_map()
for t in range(T):
    E = {(i,j): distance(x[i],x[j]) < alpha*(r[i]+r[j])}
    for (i,j) in E:
        m = memory.get((i,j), learned_zero)
        e = edge_encoder(x[i], x[j], edge_attributes[i,j])
        a = edge_attention(e, neighboring_active_edges(i,j), node_states)
        u = concat(e, a)
        z = sigmoid(Wz@u + Uz@m + bz)
        q = sigmoid(Wq@u + Uq@m + bq)
        candidate = tanh(Wm@u + Um@(q*m) + bm)
        m_new = (1-z)*m + z*candidate
        memory[(i,j)] = m_new
        edge_embedding[i,j] = concat(e, m_new)
    node_states = graph_message_passing(node_states, edge_embedding, E)
    prediction[t] = decoder(node_states, edge_embedding)

Use detached memory between training chunks only if memory growth causes backpropagation-through-time to exceed the budget; otherwise backpropagate for 8-32 steps. Retain inactive memories for a fixed horizon H, or evict them with an LRU policy after H steps. The attention formula above can be replaced with ordinary local edge aggregation for the first MVP.

(3) Computed from the paper's mathematics: dynamic edge construction, stable pair-key lookup, and the gated recurrence. Estimated empirically: the memory dimension d_mem, retention horizon H, attention neighborhood, and whether inactive memories should decay. Add an optional decay $m\leftarrow\rho^{\Delta t}m$ on retrieval, with learned or tuned $\rho\in(0,1]$, to prevent stale information.

(4) First cheap experiment: use a synthetic 2D multi-agent interaction dataset with 32 particles whose pair interactions switch on when distance is below a threshold, and train a 2-layer message-passing network to predict positions 10-50 steps ahead. Compare (a) no temporal memory, (b) node GRU memory, and (c) persistent edge GRU memory with identical hidden size and FLOPs. Report one-step MSE, 50-step rollout MSE, error immediately after a previously seen edge reappears, and memory/latency overhead. The hypothesis is specifically lower reactivation error and more stable long rollouts for (c), not necessarily lower one-step error. A successful signal is at least 20% lower long-horizon MSE at equal parameter count, with less than 30% inference overhead.