Implicit Higher-Order TPR Memory

Integrate this at the memory or retrieval layer of a Transformer, relational graph network, or object-centric world model. Input objects are F[B,N,R,D]; a conjunction query contains m pairs of role indices a_k and filler queries q_f[k]. Do not construct the paper's explicit order-j tensor, whose storage grows exponentially with tensor order. Instead compute one scalar contraction per query factor: c[k,n] = dot(normalize(q_f[k]), normalize(F[:,n,a_k])). Form the higher-order score score[n] = product_k c[k,n]. For numerical stability, use log_score[n] = sum_k log(max(c[k,n], epsilon)) when similarities are nonnegative, or retain sign bits and sum log(abs(c[k,n]) + epsilon) when signed fillers are required. Normalize with a = softmax(score / tau) and extract the requested output role using out = einsum('bn,bnd->bd', a, F[:,:,target_role,:]). For multiple heads, assign separate conjunction queries and concatenate outputs. The paper provides the higher-order contraction factorization; empirical choices are filler normalization, epsilon, temperature, and whether to use products or sums of log similarities. First cheap experiment: synthetic set reasoning with 4 roles, 8 fillers per role, 32 to 256 objects, and two- or three-factor conjunction queries. Compare dense flattened high-order attention, ordinary single-factor attention, and this factorized module at equal retrieval FLOPs while recording peak memory. Pre-register that peak memory grows linearly with object count, three-factor held-out conjunction accuracy exceeds single-factor attention by at least 15 points, and the top-1 minus top-2 score margin increases by at least 20 percent from two to three factors. Ablate only the product factorization by replacing it with an MLP over concatenated factor similarities. Falsification is superlinear memory, no conjunction-specific accuracy improvement, or a non-increasing score margin.