{
 "artifacts": null,
 "category": "regularization",
 "description": "Interpret antisymmetric token-to-token interactions as a discrete connection curvature and penalize excessive curvature rather than smoothing all attention logits. This preserves directional priority patterns while discouraging unstable, rapidly changing attention fields.",
 "formulas_latex": [
  "$$E(A,\\phi)=\\int_M\\|D_A\\phi\\|^2+\\int_M\\|F_A\\|^2$$",
  "$$B_{ij}:=\\frac{q_i^{\\mathsf T}Wk_j}{\\sqrt{d_h}},\\qquad F_{ij}:=\\frac{1}{2}(B_{ij}-B_{ji})$$",
  "$$\\mathcal{L}_{\\mathrm{total}}=\\mathcal{L}_{\\mathrm{task}}+\\lambda_F\\frac{1}{|\\Omega|}\\sum_{(i,j)\\in\\Omega}F_{ij}^2$$",
  "$$A_{ij}=\\operatorname{softmax}_j\\!\\left(B_{ij}/\\tau\\right)$$"
 ],
 "id": 317,
 "implementation": "Modify the attention block immediately after computing query and key projections and before adding the causal or padding mask. For each head, calculate B=QWK^T/sqrt(d_h), where Q and K have shape T by d_h and W is either the identity or a learned matrix initialized to identity. Construct F=(B-B^T)/2 and add lambda_F*mean(F_ij^2) over valid token pairs to the task loss. The forward attention remains A=softmax(B/tau+M), with the usual mask M; this is a curvature regularizer, not a replacement for softmax. For causal attention, compute the penalty on randomly sampled unmasked ordered pairs, or on pairs valid in both directions, so the loss does not require a second full attention matrix. Estimate the penalty exactly for short sequences and with pair subsampling for long sequences. Sweep lambda_F over logarithmic values and optionally warm it up from zero over the first 10 percent of training, since forcing symmetric logits immediately may remove useful directional structure. The cheapest experiment is a six-layer small Transformer on WikiText-2 and a synthetic long-range copy task, compared with standard attention and an explicitly symmetric-logit baseline at equal FLOPs. Measure validation perplexity, gradient-norm variance, attention entropy, and loss spikes under sequence-length extrapolation. Success means fewer optimization instabilities or better long-context perplexity at the same parameter count without collapse to uniform attention.",
 "math_summary": "The paper proposes a Yang-Mills-Higgs energy consisting of a covariant section term and a curvature term, \\(\\int_M\\|D_A\\phi\\|^2+\\int_M\\|F_A\\|^2\\), where \\(\\phi\\) is the classifier section, \\(A\\) is a connection, \\(D_A\\phi\\) is its covariant derivative, and \\(F_A\\) is the connection curvature 2-form. It identifies \\(F_A(x)\\) with the antisymmetric component of an attention bilinear at position x, with \\(\\|F_A(x)\\|^2\\) measuring geometric interaction content. For an implementable discrete analogue, let \\(q_i,k_i\\in\\mathbb{R}^{d_h}\\) be the query and key of token i, let \\(W\\in\\mathbb{R}^{d_h\\times d_h}\\) be a learned bilinear matrix, and define \\(B_{ij}=q_i^TWk_j/\\sqrt{d_h}\\). Its antisymmetric curvature proxy is \\(F_{ij}=\\tfrac12(B_{ij}-B_{ji})\\). The regularizer is the Frobenius Yang-Mills energy \\(\\|F\\|_F^2=\\sum_{i,j}F_{ij}^2\\), normalized by the number of valid token pairs.",
 "math_tags": [
  "differential-geometry",
  "geometry",
  "lie-groups"
 ],
 "ml_areas": [
  "attention",
  "transformer",
  "regularization",
  "training-dynamics"
 ],
 "paper": {
  "arxiv_id": "2607.00999",
  "arxiv_url": "https://arxiv.org/abs/2607.00999",
  "summary_what_math_gives_to_ml": "The paper provides a concrete way to treat binary labels as sections of a possibly twisted Z2 bundle rather than as a globally defined sign function. Its transferable asset is the separation between local label compatibility and global monodromy: cycle products can encode unavoidable parity obstructions on periodic or graph-structured data, while gauge-equivalent choices of reference paths represent the same classifier. A practical neural implementation is a topology-aware graph or sequence head that learns binary edge transports and predicts logits in a twisted local frame, with explicit cycle-consistency and covariant-smoothness losses. A second, more exploratory transfer is to interpret antisymmetric attention interactions as a discrete curvature and regularize their Yang-Mills energy.",
  "title": "Yang-Mills-Higgs: A Geometric Theory of Binary Labels on Non-Contractible Spaces",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 4,
  "novelty": 6,
  "usefulness": 5
 },
 "solves": [
  "stability",
  "generalization",
  "accuracy"
 ],
 "title": "Yang-Mills curvature penalty for attention",
 "url": "https://synthcore.org/idea/317/yang-mills-curvature-penalty-for-attention",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}