Cocycle-Twisted Attention

Integrate the modification into the value-composition line of a Transformer self-attention block or into edge-message aggregation in a graph neural network. Give every token or node i a grade a_i in a small finite group A, obtained from known structured labels, a learned classifier, or synthetic metadata. Compute ordinary queries, keys, and values. For an edge from j to i, use the usual attention coefficient alpha_ij and multiply the value contribution by omega(a_i,a_j). Start with real signs omega in {+1,-1}; this avoids requiring complex-valued neural layers. For multi-hop composition, maintain an accumulated grade g and phase s. When appending a message with grade b, update s <- s * omega(g,b), then update g <- g b. The identity omega(a,b) omega(ab,c) = omega(b,c) omega(a,bc) ensures that the phase is invariant to whether three grades are grouped as (a b) c or a (b c). Use omega = 1 as the exact baseline. Compare the constrained cocycle layer against unconstrained learned pairwise grade gates with the same parameter count. The paper supplies the coherence equations and the local braiding factor; the initial omega table can be a fixed nontrivial cocycle, while a learned version can optimize a phase table and periodically project it onto the cocycle constraints. The first experiment should use a 4- or 8-element cyclic group, a synthetic multi-step composition or parity task, and a 2-layer 4-head Transformer. Record validation accuracy, seed variance, loss under reordered computation, and FLOPs. A successful result is higher accuracy and lower seed variance at equal compute, especially on examples requiring three or more relational compositions.