{
 "artifacts": null,
 "category": "architecture",
 "description": "Replace ordinary row-softmax attention with a doubly stochastic Sinkhorn attention plan W, and periodically recover a gauge-fixed pairwise cost from W using the exact double-centering identity. Use this recovered cost to initialize or regularize a structured attention score, making the attention geometry identifiable despite arbitrary query and key row and column offsets.",
 "download_zip": "https://synthcore.org/download/1163",
 "formulas_latex": [
  "$$W=\\operatorname{diag}(u)K\\operatorname{diag}(v),\\qquad K_{ij}=\\exp\\left(-\\frac{C_{ij}}{\\varepsilon}\\right)$$",
  "$$\\log W=\\log(u)\\mathbf{1}^{\\top}+\\mathbf{1}\\log(v)^{\\top}-\\frac{C}{\\varepsilon}$$",
  "$$H_m(\\log W)H_n=-\\frac{1}{\\varepsilon}H_mCH_n,\\qquad H_q=I_q-\\frac{1}{q}\\mathbf{1}\\mathbf{1}^{\\top}$$",
  "$$\\widehat C=-\\varepsilon H_m\\log(W+\\delta)H_n$$"
 ],
 "id": 3000,
 "implementation": "Integrate this into a cross-attention block. Convert attention scores S into a positive kernel K with K_ij = exp(-S_ij divided by epsilon), then run 5 to 20 log-domain Sinkhorn iterations to obtain W with prescribed source and target marginals. Keep value aggregation Y = W V, but add an inverse-cost consistency term or use the recovered cost as a structured bias for the next attention block. The pseudocode is: compute stabilized K; initialize positive u and v; repeat u = s divided elementwise by K v and v = r divided elementwise by K-transpose u; form W = diag(u) K diag(v); set L = log(W plus delta); compute C_inv = -epsilon H_m L H_n; add lambda times the squared Frobenius norm of H_m(S - C_inv)H_n to the training loss. The exact inverse identity is supplied by the paper. The engineer must estimate the effect of finite Sinkhorn convergence, floating-point flooring, and model mismatch. First run a synthetic test with random C, prescribed marginals, and generated W before training a two-layer Transformer on a small copy or patch-classification task. Compare softmax attention with Sinkhorn attention with and without the inverse-cost penalty. The prediction is that the relative centered reconstruction error will remain below 5 percent when the smallest W entry exceeds 10^-5 and Sinkhorn marginal error is below 10^-4. Adding any row-plus-column gauge to C must not change C_inv. Reconstruction error should increase sharply when W entries fall below delta.",
 "math_summary": "For source and target marginals s and r, entropic optimal transport has coupling W = diag(u) K diag(v), where K_ij = exp(-C_ij divided by epsilon), C is the cost matrix, epsilon is the entropic temperature, and u and v are Sinkhorn scaling vectors. Consequently, log W = log(u) 1-transpose + 1 log(v)-transpose - C divided by epsilon. Let H_m = I_m - 1 1-transpose divided by m and H_n = I_n - 1 1-transpose divided by n be centering matrices. Double-centering removes the row and column gauge terms, yielding H_m log(W) H_n = -H_m C H_n divided by epsilon. The recoverable gauge-fixed cost is therefore C_hat = -epsilon H_m log(W plus delta) H_n, where delta is a positive numerical floor. Costs differing by a 1-transpose plus 1 b-transpose generate the same transport plan, so only the double-centered cost is identifiable.",
 "math_tags": [
  "optimal-transport",
  "linear-algebra",
  "information-theory"
 ],
 "ml_areas": [
  "attention",
  "transformer",
  "regularization"
 ],
 "paper": {
  "arxiv_id": "2609.01278",
  "arxiv_url": "https://arxiv.org/abs/2609.01278",
  "summary_what_math_gives_to_ml": "The paper provides an exact inverse map for entropic optimal transport: from an observed coupling W, the cost matrix can be recovered up to the unavoidable row-plus-column gauge by applying double-centering to log W. This removes the Sinkhorn scaling potentials without iterative optimization, and the same structure permits estimating the entropic temperature when some true cost entries are known. The strongest neural-network transfer is a Sinkhorn-constrained attention or routing layer whose observed transport plan is converted into a gauge-fixed cost matrix for initialization, distillation, monitoring, or adaptive temperature control. The key falsifiable signature is exact affine recovery of centered costs, with degradation only when entries of W approach numerical zero.",
  "title": "An exact and fast solution of the inverse Regularized Optimal Transport problem",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 7,
  "usefulness": 7
 },
 "solves": [
  "stability",
  "accuracy",
  "generalization"
 ],
 "title": "Gauge-Free Inverse OT Attention",
 "url": "https://synthcore.org/idea/3000/gauge-free-inverse-ot-attention",
 "verification": {
  "peer_reviewed": false,
  "status": "queued_mechanism",
  "status_label": "Queued — mechanism check",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}