{
 "artifacts": [
  {
   "name": "bench_pgw.py",
   "url": "https://synthcore.org/code/582/bench_pgw.py"
  },
  {
   "name": "bench_report.json",
   "url": "https://synthcore.org/code/582/bench_report.json"
  },
  {
   "name": "experiment.py",
   "url": "https://synthcore.org/code/582/experiment.py"
  },
  {
   "name": "report.md",
   "url": "https://synthcore.org/code/582/report.md"
  },
  {
   "name": "report_bench_2026-08-31T184541.md",
   "url": "https://synthcore.org/code/582/report_bench_2026-08-31T184541.md"
  },
  {
   "name": "results.json",
   "url": "https://synthcore.org/code/582/results.json"
  }
 ],
 "category": "architecture",
 "description": "Replace unconstrained softmax cross-attention with a many-to-many transport matrix whose row and column masses have explicit upper bounds. Compute the attention cost from both feature similarity and pairwise relational disagreement, so a token is attended to only when its relationships to other tokens are jointly compatible. The inequality constraints provide a principled dustbin-free mechanism for ignoring distractor tokens.",
 "download_zip": "https://synthcore.org/download/582",
 "formulas_latex": [
  "$$\\mathcal{C}_{1}(\\mu):=\\{T\\in[0,1]^{m\\times n}\\mid T\\mathbbm{1}_{n}\\preceq\\mu\\},\\qquad \\mathcal{C}_{2}(\\nu):=\\{T\\in[0,1]^{m\\times n}\\mid T^{\\top}\\mathbbm{1}_{m}\\preceq\\nu\\}.$$",
  "$$T\\in\\mathcal{C}_{1}(\\mu)\\cap\\mathcal{C}_{2}(\\nu),\\qquad T_{ij}\\text{ represents the alignment between species }i\\text{ and }j.$$",
  "$$\\min_{T\\ge 0}\\;\\sum_{i=1}^{m}\\sum_{j=1}^{n}T_{ij}\\bigl[\\alpha c_{ij}+\\beta L_{ij}(T)\\bigr]-\\varepsilon H(T),\\quad L_{ij}(T)=\\sum_{k=1}^{m}\\sum_{l=1}^{n}|A_{ik}-B_{jl}|^{2}T_{kl},\\quad H(T)=-\\sum_{i,j}T_{ij}(\\log T_{ij}-1).$$"
 ],
 "id": 1491,
 "implementation": "Integrate this as a cross-attention replacement between two variable-length sets: source tokens X = [x_1,...,x_m] and target tokens Y = [y_1,...,y_n], rather than modifying the value projection. First normalize masses to uniform or learned positive distributions mu and nu; assign padded positions zero mass. Compute feature costs c_ij = 1 - cosine(W_q x_i, W_k y_j). Build relational matrices A and B using pairwise distances of projected features, k-nearest-neighbor distances, graph adjacency, or relative positions. Initialize T as a positive matrix satisfying the upper bounds. Repeat 3–10 iterations: calculate L_ij(T) = sum over k,l of |A_ik - B_jl|^2 T_kl; form logits S_ij = -(alpha c_ij + beta L_ij(T))/epsilon; set a positive kernel K_ij = exp(S_ij); alternately rescale rows and columns so row sums do not exceed mu and column sums do not exceed nu, clipping every scaling factor at one; and optionally renormalize by transported mass. Use output o_i = sum_j T_ij V_j divided by sum_j T_ij plus delta, with delta preventing division by zero. Optionally concatenate the unmatched gate u_i = 1 - sum_j T_ij. During training, unroll a fixed number of iterations and backpropagate through them; for a cheap MVP, stop gradients through L and use it only to update T. Estimate A and B directly from features, not labels. Start on synthetic 2-D point clouds or graph-node sets where one set contains 30–50% distractors and the ground-truth transformation preserves pairwise distances. Compare against dot-product softmax attention and balanced Sinkhorn attention in a small Transformer or graph matcher. Measure correspondence accuracy, retrieval accuracy with distractors, transported-mass selectivity, attention entropy, gradient stability, and wall-clock cost. Success means higher matching accuracy at equal parameter count and lower mass assigned to distractors, with no more than roughly 2x attention-layer runtime.",
 "math_summary": "Definition 2.1 defines a non-deterministic alignment between networks as a matrix T in C_1(mu) intersect C_2(nu), where T_ij is the alignment mass between source item i and target item j, mu in Delta^(m-1) and nu in Delta^(n-1) are source and target importance distributions, m and n are the set sizes, and 1_k is an all-ones vector. The constraints T 1_n \u003c= mu and T^T 1_m \u003c= nu mean that each source and target can allocate at most its available mass; unlike balanced optimal transport, total transported mass may be smaller than one, and unlike a permutation, one item may align with many items. For neural features x_i and y_j, use feature dissimilarity c_ij = 1 - cosine(x_i,y_j). Let A_ik and B_jl be pairwise relational distances or similarities within the two sets. A Gromov-Wasserstein-inspired structural cost is L_ij(T) = sum over k,l of |A_ik - B_jl|^2 T_kl. Optimize the partial entropic objective sum over i,j of T_ij times [alpha c_ij + beta L_ij(T)] minus epsilon H(T), where alpha and beta are nonnegative feature and structure weights, epsilon is the entropy temperature, and H(T) = -sum over i,j T_ij(log T_ij - 1). The structural term rewards alignments that preserve relations rather than merely matching individual embeddings.",
 "math_tags": [
  "optimal-transport",
  "graph-theory",
  "geometry",
  "convex-analysis"
 ],
 "ml_areas": [
  "attention",
  "graph-nn",
  "transformer"
 ],
 "paper": {
  "arxiv_id": "2607.27496",
  "arxiv_url": "https://arxiv.org/abs/2607.27496",
  "summary_what_math_gives_to_ml": "The paper contributes a capacity-constrained, non-deterministic alignment: an alignment is a nonnegative matrix whose row and column sums are upper-bounded by importance distributions, so objects may remain unmatched while other objects participate in many-to-many correspondences. Combined with Gromov–Wasserstein-style relational costs, this is a useful alternative to ordinary attention, which matches tokens using local query-key similarity without explicit capacity or structural-consistency constraints. The most promising neural transfer is a partial structural cross-attention layer for matching variable-sized token, patch, graph-node, or memory sets while preserving pairwise relations and allowing irrelevant elements to be dropped. A first experiment should compare this layer with softmax cross-attention on noisy graph matching or multimodal retrieval, measuring alignment precision, robustness to distractors, and computational overhead.",
  "title": "Identifying common backbones of interactions underlying food webs via non-deterministic alignments",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 6,
  "novelty": 5,
  "usefulness": 7
 },
 "solves": [
  "accuracy",
  "stability",
  "scalability"
 ],
 "title": "Partial Gromov-Wasserstein Cross-Attention",
 "url": "https://synthcore.org/idea/1491/partial-gromov-wasserstein-cross-attention",
 "verification": {
  "benchmark": {
   "track": "sequence",
   "model": "transformer_tiny",
   "n_seeds": 8,
   "metric_direction": "lower is better",
   "comparison": {
    "baseline_mean": 0.6229188442230225,
    "idea_mean": 0,
    "delta_mean": -0.1980590299,
    "per_seed_diffs": [
     -0.2143552601,
     -0.2253650129,
     -0.1206991374,
     -0.2842617631,
     -0.1242177784,
     -0.2546616197,
     -0.1158161461,
     -0.2450955212
    ],
    "idea_wins": 8,
    "n_pairs": 8,
    "p_value": 0.0081,
    "mde": 0,
    "mde_rel_pct": null,
    "verdict": "idea better (significant)",
    "system_worked": true
   },
   "baseline": {
    "best_cfg": {
     "lr": 0.0015
    },
    "sweep": [
     {
      "cfg": {
       "lr": 0.0015
      },
      "mean": 0.6402924656867981
     },
     {
      "cfg": {
       "lr": 0.003
      },
      "mean": 0.7056357711553574
     },
     {
      "cfg": {
       "lr": 0.006
      },
      "mean": 0.9283334761857986
     }
    ],
    "full": {
     "mean": 0.6229188442230225,
     "std": 0.06931051155059516,
     "per_seed": [
      0.7044264674,
      0.5437753201,
      0.5856346488,
      0.7273334265,
      0.5151234269,
      0.6567925215,
      0.610052824,
      0.6402121186
     ],
     "n": 8
    }
   },
   "idea": {
    "mean": 0,
    "std": 0,
    "per_seed": [
     0.4900712073,
     0.3184103072,
     0.4649355114,
     0.4430716634,
     0.3909056485,
     0.4021309018,
     0.4942366779,
     0.3951165974
    ],
    "n": 0
   },
   "mechanism_signature": {
    "confirmed": false,
    "observed": {
     "baseline_test_mse_mean": 0.6229188442,
     "idea_test_mse_mean": 0.4248598143
    },
    "predicted": {
     "beta": 0.2,
     "partial_row_upper_bound": 0.03125
    },
    "prediction": "relational compatibility reduces incompatible-token attention"
   },
   "custom_track": null
  },
  "mechanism_confirmed": false,
  "peer_reviewed": false,
  "practical_verdict": "helps",
  "stage1_mechanism_check": {
   "worked": true,
   "confidence": 8,
   "verdict": "Built a readable partial GW cross-attention MVP with cosine feature cost, relational GW cost, entropic kernels, and alternating clipped row/column upper-bound projections. Quantitative checks matched predictions: beta=0 relative invariance error was 0, convex linearity error was 3.33e-16, beta logit shifts scaled exactly from 0 to 21.13, and maximum feasibility violations were 0 for all tested shapes. In the synthetic benchmark, increasing beta improved matched mass from 0.717 to 0.882 and reduced distractor mass from 0.151 to 0.090, while accuracy stayed 1.0; however, partial GW did not beat softmax accuracy and assigned slightly more distractor fraction than softmax, so no general performance win is established.",
   "metrics": {
    "baseline": "Softmax: accuracy 1.000, matched_mass_fraction 0.748, distractor_mass_fraction 0.106, transported_mass 12.0, entropy 0.769",
    "idea": "Partial GW: accuracy 1.000, matched_mass_fraction 0.800, distractor_mass_fraction 0.123, transported_mass 0.684, entropy 3.416; beta sweep matched mass 0.717-\u003e0.882 and distractor fraction 0.151-\u003e0.090 for beta 0-\u003e3"
   },
   "how_to_run": "python3 experiment.py",
   "files": [
    "experiment.py",
    "results.json"
   ],
   "limitations": "Only a small synthetic feature-matching benchmark was tested; no learned neural network, backpropagation, variable padding masks, learned masses, runtime/FLOP profiling, or larger/noisier real-world correspondence task was evaluated. The beta linearity and feasibility checks validate algebraic properties directly and are not evidence of end-to-end accuracy gains."
  },
  "status": "beats_baseline",
  "status_label": "Beats tuned baseline",
  "updated_at": "2026-09-02T12:30:10",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": false,
    "tested": true
   },
   "practical_benchmark": {
    "beats_baseline": true,
    "tested": true,
    "verdict": "helps"
   },
   "toy_mechanism_gate": {
    "confirmed": true,
    "tested": true
   }
  }
 }
}