{
 "artifacts": null,
 "category": "architecture",
 "description": "Use the adjacency matrix of a vertex-transitive strongly regular graph as a fixed sparse attention or token-mixing mask. Every vertex has the same degree, and every pair of vertices has exactly one of two common-neighbor counts, giving predictable two-hop coverage and avoiding the degree and connectivity irregularities of random sparsification.",
 "formulas_latex": [
  "$$X^{2}-\\left(\\frac{v}{2}+a\\right)X+\\frac{a}{2}(v-1)=0,\\qquad \\lambda=k+\\frac{a-v}{2},\\qquad \\mu=k-\\frac{a}{2}.$$",
  "$$\\mathcal{T}_{v}-v\\;\\text{is an }srg\\left(v-1,a,\\frac{3a-v}{2},\\frac{a}{2}\\right).$$",
  "$$P=\\frac{A+I}{k+1},\\qquad H^{(\\ell+1)}=\\sigma\\!\\left(PH^{(\\ell)}W^{(\\ell)}\\right)+H^{(\\ell)}.$$",
  "$$A^{2}=(\\lambda-\\mu)A+(k-\\mu)I+\\mu\\mathbf{1}\\mathbf{1}^{\\mathsf T}.$$"
 ],
 "id": 2885,
 "implementation": "1. Integration point: replace dense token-token attention in a small Transformer encoder, or dense token mixing in an MLP-Mixer, with fixed sparse graph aggregation. For sequence length N, choose a strongly regular graph adjacency matrix A of size N, include I for self-information, and use the same mask in every layer initially. Do not materialize N squared logits. Gather only the k graph neighbors plus the token itself. 2. Pseudocode: inputs X, sparse edge list E from A, and weights W_Q,W_K,W_V; for each layer compute Q=XW_Q, K=XW_K, V=XW_V; for every edge (i,j) in E union self-loops compute l_ij=Q_i K_j^T/sqrt(d_model); set alpha_ij=exp(l_ij)/sum_{r in N(i) union {i}} exp(l_ir); compute Y_i=sum_j alpha_ij V_j; then update X=X+W_OY followed by the normal feed-forward block. The cheapest ablation uses fixed alpha_ij=1/(k+1), implementing P directly. 3. Compute from the paper: solve the displayed quadratic for candidate k from v and a, derive lambda and mu, and verify the graph with A^2=(lambda-mu)A+(k-mu)I+mu 11^T. Estimate no graph quantities during training, but measure empirical two-hop coverage and the eigenvalues of P. 4. First experiment: train a 6-layer CIFAR-10 ViT or a character-level Transformer at sequence length 128 using dense attention, a random k-regular mask, a random expander mask, and the strongly regular mask at equal k and hidden width. Measure validation accuracy or perplexity, loss versus training FLOPs, wall-clock throughput, and activation memory. Success means roughly N/k attention-cost scaling with no accuracy loss, faster or more stable early loss descent than random regular masks, and performance that is insensitive to token permutation because vertex transitivity removes mask-location artifacts.",
 "math_summary": "A strongly regular graph srg(N,k,lambda,mu) has N vertices, degree k, lambda common neighbors for every adjacent pair, and mu common neighbors for every nonadjacent pair. The paper states that for a regular two-graph with parameter a, every descendant with one vertex removed is srg(v-1,a,(3a-v)/2,a/2), and that any srg(v,k,lambda,mu) in the switching class has degree k satisfying k^2-(v/2+a)k+(a/2)(v-1)=0, with lambda=k+(a-v)/2 and mu=k-a/2. We exploit the resulting vertex-uniform degree and exact two-hop overlap rather than the classification itself. For an attention mask A, let A_ij=1 when vertices i and j are adjacent, let I be the self-loop matrix, and let d=k+1 be the masked degree including self-attention. Vertex transitivity makes every row degree equal, so the normalized propagation operator is P=(A+I)/(k+1).",
 "math_tags": [
  "graph-theory",
  "combinatorics",
  "algebra"
 ],
 "ml_areas": [
  "attention",
  "transformer",
  "graph-nn",
  "inference-speedup"
 ],
 "paper": {
  "arxiv_id": "2608.30330",
  "arxiv_url": "https://arxiv.org/abs/2608.30330",
  "summary_what_math_gives_to_ml": "The paper supplies highly symmetric sparse graphs obtained from switching classes of regular two-graphs, with exact strongly regular parameters controlling every pairwise common-neighbor count. This uniformity is transferable to sparse neural connectivity: unlike an arbitrary k-regular mask, the resulting interaction graph is vertex-transitive and has only two possible two-hop overlap patterns, which can improve information mixing without hubs. The most direct experiment is to replace dense self-attention or MLP token mixing by a fixed strongly regular graph mask while preserving residual and local feature projections, and compare against random regular and expander masks at equal edge count.",
  "title": "Vertex-transitive strongly regular graphs in the switching class of doubly transitive two-graphs",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 6,
  "usefulness": 5
 },
 "solves": [
  "speedup",
  "scalability",
  "stability"
 ],
 "title": "Strongly-regular sparse attention",
 "url": "https://synthcore.org/idea/2885/strongly-regular-sparse-attention",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}