Balanced design attention MVP

What is implemented

balanced_design_attention.py implements:

• cyclic exact designs for the Fano (v,b,k,r,lambda)=(7,7,3,3,1) design and the projective-plane (13,13,4,4,1) design;
• incidence-matrix construction and checks of MM^T=(r-lambda)I+lambda J;
• signed membership vectors Z(x,y)=M^T(e_x-e_y);
• block attention with gather, local softmax attention, scatter, and per-token averaging;
• dense, contiguous, random, and exact-design attention timing comparisons.

Stage-1 quantitative predictions and checks

1. Incidence identity / uniform coverage: for an exact design, every row degree is r and every off-diagonal pair co-occurrence is lambda, so the maximum absolute error in the matrix identity and both variances should be zero.

   • Fano design: predicted row degree 3, pair coverage 1, observed row range 3..3, pair range 1..1, identity error 0, row variance 0, pair variance 0.
   • Projective-13 design: predicted row degree 4, pair coverage 1, observed row range 4..4, pair range 1..1, identity error 0, row variance 0, pair variance 0.

2. Signed-membership norm: every pair should satisfy ||Z(x,y)||^2=2(r-lambda).

   • Fano: prediction 4; observed min/max/mean 4/4/4.
   • Projective-13: prediction 6; observed min/max/mean 6/6/6.

3. Replication scaling: repeating every block s times predicts r -> 3s, lambda -> s, and ||Z||^2 -> 4s, with zero coverage variance preserved.

   • s=1,2,4,8: observed norm squared 4,8,16,32, exactly equal to prediction; row and pair coverage variances remained 0 in every case.

These checks are exact integer computations, not approximate tolerance matches.

Mini benchmark

CUDA was used, with fixed seed 123, feature width 32, and complete designs on their natural point sets. The sparse implementation uses a Python loop over blocks, so these times measure this MVP rather than an optimized fused kernel.

| n | method | score FLOPs | median ms | pair variance | row-degree variance | |---:|---|---:|---:|---:|---:| | 7 | dense | 3,136 | 2.305 | 0 | 0 | | 7 | contiguous | 4,032 | 14.092 | 0.667 | 0 | | 7 | random | 4,032 | 15.282 | 0.952 | 2.000 | | 7 | exact design | 4,032 | 15.880 | 0 | 0 | | 13 | dense | 10,816 | 0.298 | | | 13 | contiguous | 13,312 | 30.297 | 1.333 | 0 | | 13 | random | 13,312 | 18.884 | 0.769 | 2.308 | | 13 | exact design | 13,312 | 22.086 | 0 | 0 |

The exact designs clearly deliver the promised balanced connectivity. However, this tiny unoptimized benchmark does not demonstrate a wall-clock speedup: Python scatter/index operations dominate, and the natural small designs have bk^2 greater than v^2 for these parameters. No language-model perplexity or long-range copy task was tested.

Reproduction

/home/maxwelhelp/main/bin/python3 balanced_design_attention.py

The command writes results.json and prints the same JSON report.