{
 "artifacts": [
  {
   "name": "polar_slack.py",
   "url": "https://synthcore.org/code/1228/polar_slack.py"
  },
  {
   "name": "report.md",
   "url": "https://synthcore.org/code/1228/report.md"
  },
  {
   "name": "run_experiment.py",
   "url": "https://synthcore.org/code/1228/run_experiment.py"
  }
 ],
 "category": "architecture",
 "description": "Use a spherical-design codebook and the paper's polar slack factorization to create a nonnegative geometric interaction bias for attention or expert routing. The resulting kernel is generated by a rank-one term and a rank-at-most-d term, and entries close to zero can define a structured sparse mask instead of relying only on learned top-k selection.",
 "download_zip": "https://synthcore.org/download/1228",
 "formulas_latex": [
  "$$P_X^{\\circ}=\\operatorname{conv}\\{x_i/h_i:1\\le i\\le N\\}.$$",
  "$$A=c\\,hh^{T}-X^{T}U^{T}X,\\qquad A\\ge 0,\\qquad \\operatorname{rank}A=d+1.$$",
  "$$A_{ij}=c\\,h_i h_j-x_i^{T}U^{T}x_j.$$",
  "$$S_{ij}=\\frac{q_i^{T}k_j}{\\sqrt{d_k}}+\\alpha\\log(A_{ij}+\\varepsilon),\\qquad \\operatorname{Attn}(Q,K,V)_i=\\sum_j\\operatorname{softmax}_j(S_{ij})V_j.$$"
 ],
 "id": 3055,
 "implementation": "Integrate the construction into one attention layer or an MoE router whose sequence positions or experts are indexed by N geometric slots. Construct a spherical-design matrix X with N columns in R^d and normalize every column to unit norm. Store positive support values h_i using h_i = exp(g_i), where g_i is fixed initially or is a trainable scalar. Store U as an orthogonal matrix, initially U = I; if it is trained, parameterize it with a Cayley transform or periodically project the raw matrix onto O(d) using QR. At initialization compute rho = max_{i,j} x_i^T U^T x_j and choose c = (1 + margin) rho divided by min_{i,j}(h_i h_j), with margin such as 0.01; if rho is nonpositive, use c = 1. Recompute A = c outer(h,h) - X^T U^T X on every forward pass for small N, or cache it and refresh it periodically for larger N. Clamp A to max(A,0), form B = log(A + epsilon), and add alpha B to the standard QK^T divided by sqrt(d_k). For sparse inference, keep only the k largest A entries in each row or retain entries with A_{ij} greater than a threshold tau, then apply a sparse softmax over the retained indices. The paper-derived quantities are the factorized slack formula, its nonnegativity target, and its rank structure; alpha, epsilon, k, tau, and whether h and U are trainable are empirical choices. First run a small experiment with a 2-layer, 4-head Transformer on CIFAR-10 converted to patches or a character-level language model on Tiny Shakespeare. Compare dense attention, ordinary top-k attention, and geometric slack attention at equal hidden size. Measure validation loss, active edges per row, attention FLOPs, GPU memory, and seed-to-seed variance. The method succeeds if it reaches the same validation loss with at least 2x fewer retained attention edges, or gives faster loss reduction at equal FLOPs without producing negative-kernel numerical failures.",
 "math_summary": "Let X = [x_1,...,x_N] be a d by N matrix whose columns x_i are unit spherical-design points, let h = (h_1,...,h_N)^T contain positive support numbers, let U be an orthogonal d by d matrix, and let c \u003e 0. Under the paper's self-polar relation, the slack matrix is A = c h h^T - X^T U^T X, with A_{ij} = c h_i h_j - x_i^T U^T x_j. The paper states that A is nonnegative, has rank d+1, and has a zero pattern equal to facet-vertex incidence. The rank bound follows because c h h^T has rank one and X^T U^T X has rank at most d. For neural attention, use A as a positive structural kernel and add log(A + epsilon) to ordinary query-key logits, where epsilon \u003e 0 avoids logarithms of zero. The mathematical guarantee applies only when h, c, and U form the paper's valid polar realization; in an ML relaxation, nonnegativity is enforced explicitly and the factorization is retained even when exact incidence zeros are absent.",
 "math_tags": [
  "geometry",
  "convex-analysis",
  "linear-algebra"
 ],
 "ml_areas": [
  "attention",
  "moe-routing",
  "inference-speedup",
  "memory"
 ],
 "paper": {
  "arxiv_id": "2609.02439",
  "arxiv_url": "https://arxiv.org/abs/2609.02439",
  "summary_what_math_gives_to_ml": "The paper offers a transferable construction linking spherical designs, polar convex bodies, and structured nonnegative slack matrices. The most promising neural-network use is the factorization A = c hh^T - X^T U^T X, which creates a geometrically constrained interaction kernel from rank-one and low-rank terms while preserving nonnegativity and a meaningful zero pattern. This can replace or bias dense attention and mixture-of-experts routing, with near-zero slack values providing a principled sparse connectivity rule. A lower-risk application is using spherical-design directions as deterministic, moment-matched prototype or attention-head initialization.",
  "title": "Dual Geometry of Spherical Designs: Polarity, Self-Polar Rigidity, and Quadrature Structure",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 6,
  "novelty": 7,
  "usefulness": 7
 },
 "solves": [
  "speedup",
  "stability",
  "memory"
 ],
 "title": "Polar Slack Attention",
 "url": "https://synthcore.org/idea/3055/polar-slack-attention",
 "verification": {
  "peer_reviewed": false,
  "stage1_mechanism_check": {
   "worked": false,
   "confidence": 9,
   "verdict": "Built a readable polar-slack module and deterministic verification experiment. For the tetrahedral spherical 2-design with U=-I, h=1, and c=1/3, A is effectively (4/3)I, has three incidence zeros per row, rank 4=d+1, and the Gram term has rank 3 with singular values 4/3. The geometric log-bias gives perfect target selection in the toy proxy, but ordinary top-1 already retains one edge per row, so no improvement over that baseline was observed; arbitrary supports also produced negative slack entries.",
   "metrics": {
    "baseline": "Dense attention: target probability 0.24655, cross-entropy 1.72943, 4 edges/row; ordinary top-1 accuracy 0.24338, 1 edge/row.",
    "idea": "Polar slack: target probability 0.99999994, cross-entropy 6.02e-08, argmax accuracy 1.0, 1 effective positive edge/row after tolerance; exact slack min -1.11e-16 from roundoff, rank 4."
   },
   "how_to_run": "python3 run_experiment.py",
   "files": [
    "polar_slack.py",
    "run_experiment.py"
   ],
   "limitations": "No trained Transformer, CIFAR-10, language-model, GPU speed, memory, FLOP, or seed-to-seed training comparison was run. The exact tetrahedral case is highly structured and makes the geometric mask essentially an identity mask; it does not test larger spherical designs or learned h/U. The invalid-support test shows that the paper's nonnegativity guarantee cannot be assumed for unconstrained ML relaxations."
  },
  "status": "mechanism_failed",
  "status_label": "Mechanism failed",
  "updated_at": "2026-09-03T13:01:14",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": false,
    "tested": true
   }
  }
 }
}