✗ Failed on benchmark 2026

Gauge-Free Spectral OT Layer

Implementation & benchmark of arXiv:2608.13201 — Sinkhorn Linearization and the Spectral Proxy: Unifying the Statistical and Algorithmic Theory of Feature-Parameterized Inverse Optimal Transport via a Single Spectral Sandwich

Usefulness7/10

Difficulty5/10

Novelty7/10

Architecture Stability Accuracy Sample efficiency Attention Graph nn Regularization Optimal transport Linear algebra Spectral theory Convex analysis

Source paper: Sinkhorn Linearization and the Spectral Proxy: Unifying the Statistical and Algorithmic Theory of Feature-Parameterized Inverse Optimal Transport via a Single Spectral Sandwich arXiv:2608.13201 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Parameterize an entropic OT cost only in directions that can change the transport plan, removing row-plus-column potential directions that are invisible because of OT gauge invariance. Whiten the remaining feature coordinates using their empirical covariance, producing an OT layer whose identifiable parameters have substantially more uniform sensitivity.

Formulas

$$\pi(C,a,b)=\arg\min_{\pi\in\mathcal{M}(a,b)}\left\{\langle C,\pi\rangle+\varepsilon\sum_{ij}\pi_{ij}\log\pi_{ij}\right\},\qquad C_{\theta}(i,j)=-\theta^\top\varphi(i,j).$$

$$\frac{\pi_{\min}}{\varepsilon}I\preceq H_{\mathcal{T}}^{-1}\preceq\frac{\pi_{\max}}{\varepsilon}I.$$

$$\sigma_{\min}(\mathcal{J}_{\theta})\geq\frac{\pi_{\min}}{a_{\max}\varepsilon}\sqrt{\lambda_{\min}(\Sigma)}.$$

$$\widetilde\theta=(P_{\perp}\Sigma P_{\perp}+\delta I)^{-1/2}P_{\perp}\theta,\qquad C_{\widetilde\theta}(i,j)=-\widetilde\theta^\top\widetilde\varphi(i,j).$$

Mathematical statement

The entropic OT plan is $\pi(C,a,b)=\arg\min_{\pi\in\mathcal{M}(a,b)}\{\langle C,\pi\rangle+\varepsilon\sum_{ij}\pi_{ij}\log\pi_{ij}\}$, with $\mathcal{M}(a,b)=\{\pi\geq0:\pi\mathbf{1}=a,\pi^\top\mathbf{1}=b\}$. The cost is $C_{\theta}(i,j)=-\theta^\top\varphi(i,j)$, where $\theta\in\mathbb{R}^{F}$ and $\varphi(i,j)\in\mathbb{R}^{F}$. OT is invariant to adding a row and column potential, $C_{ij}\mapsto C_{ij}+r_i+s_j$; therefore the gauge kernel is $\mathcal{N}_{\Phi}=\{\theta:\Phi\theta\in\{r_i+s_j\}_{i,j}\}$, where $\Phi\in\mathbb{R}^{K^2\times F}$ has row $(i,j)$ equal to $\varphi(i,j)^\top$. The paper gives $F\leq(K-1)^2$ for the identifiable quotient and requires $\operatorname{rank}(\Sigma)=F$, where $\Sigma$ is the feature covariance or second-moment matrix. On the tangent space $\mathcal{T}$, the inverse restricted Hessian obeys $(\pi_{\min}/\varepsilon)I\preceq H_{\mathcal{T}}^{-1}\preceq(\pi_{\max}/\varepsilon)I$. The induced parameter-to-plan Jacobian $\mathcal{J}_{\theta}$ satisfies $\sigma_{\min}(\mathcal{J}_{\theta})\geq[\pi_{\min}/(a_{\max}\varepsilon)]\sqrt{\lambda_{\min}(\Sigma)}$, where $\pi_{\min}=\min_{ij}\pi_{ij}$, $\pi_{\max}=\max_{ij}\pi_{ij}$, $a_{\max}=\max_i a_i$, and $\lambda_{\min}(\Sigma)$ is the smallest feature-covariance eigenvalue. The adaptation uses the quotient projector $P_{\perp}$ and whitening matrix $(P_{\perp}\Sigma P_{\perp}+\delta I)^{-1/2}$.

Implementation notes

(1) Integration point: insert this parameterization immediately before cost construction in a differentiable Sinkhorn layer. The layer receives source features $x_i$, target features $y_j$, and a learned pair feature map $\varphi(i,j)$; it constructs $C_{ij}=-\theta^\top\varphi(i,j)$ and runs Sinkhorn. Do not optimize raw $\theta$ directly. (2) Build a gauge design matrix $G\in\mathbb{R}^{K^2\times(2K-1)}$. For pair $(i,j)$, its row is the concatenation of the one-hot row vector $e_i$ and target one-hot vector $e_j$, with one redundant target column removed. Compute a projector $P_G=G(G^\top G+\delta_gI)^{-1}G^\top$ and retain cost components orthogonal to this span. If features are represented in parameter space, compute a QR/SVD basis $U$ for the nullspace of the gauge-output map and use $\theta=Uz$. (3) Maintain $\Sigma_t=\beta\Sigma_{t-1}+(1-\beta)\Phi_t^\top\Phi_t/K^2$, where $\Phi_t$ contains pair-feature rows. In the retained basis, compute $W_t=(U^\top\Sigma_tU+\delta I)^{-1/2}$ and use $\theta=UW_tz$. (4) The gauge quotient, covariance whitening, and Jacobian lower bound come from the paper. The covariance eigenvalues are computed from minibatches; $\pi_{\min}$ and $a_{\max}$ are measured from the current plan and source marginal. (5) First experiment: compare raw versus gauge-free-whitened OT attention on a synthetic matching task and a small graph-matching benchmark with $K=32$, $F=64$, and a 4-layer encoder. Measure transport loss, validation accuracy, gradient-norm variance, the number of near-zero singular values in an empirical Jacobian, and failed runs. Success means faster loss decrease at equal FLOPs and fewer unstable runs, especially when features contain row- or column-bias directions.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: No measurable effect

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built a gauge-free, covariance-whitened entropic OT MVP with log-domain Sinkhorn and a synthetic matching comparison. The mechanism manifested: gauge perturbations changed the plan by at most 4.8e-15, five near-zero Jacobian singular values were removed leaving rank 3, and whitening condition numbers matched the ridge-aware prediction across all anisotropy levels. The idea also showed a modest optimization improvement, reaching 0.0015565 versus 0.0015634 for the raw baseline at 180 steps, though this is not evidence of a large practical win.

Agent confidence: 9/10

Baseline: Raw parameterization: final mean matching loss 0.00156339 (std 0.00007242); losses at steps 10/30/60/120/180: 0.00161616/0.00157786/0.00156467/0.00156339/0.00156339.
Idea: Gauge-free whitened parameterization: final mean matching loss 0.00155647 (std 0.00007205); losses at steps 10/30/60/120/180: 0.00159108/0.00156080/0.00155677/0.00155647/0.00155647. Gauge plan-change sweep: 0, 2.6e-17, 2.8e-17, 2.7e-16, 4.8e-15. Raw Jacobian had 5 near-zero singular values; quotient retained rank 3. Raw covariance condition rose from 1.83 to 1.12e8 as anisotropy alpha decreased from 1 to 0.01, while observed whitened conditions were 1.00, 1.00, 1.014, 2.707, and 139.3, matching the ridge-aware predictions.

Limitations: Only a tiny synthetic K=8 experiment was tested, not the proposed K=32 graph benchmark or a multi-layer attention integration. The whitening ridge limits conditioning for extremely small eigenvalues, and the optimization comparison used a single fixed learning rate and six trials rather than a tuned FLOP-matched study.

How to run: python3 run_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.13201) Sep 2, 2026

Ahead of the baseline (−0.0%) but not statistically significant (p=1, wins 2 / 8 paired seeds).

Benchmark: Sequence modeling
Model: transformer_tiny
Paired seeds: 8
Baseline mean: 1.0049
Idea mean: 1.0049
Effect (Δ): -0 (−0.0%; negative = idea better)
Wins: 2 / 8 paired seeds
p-value: 1 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±0.0%
Mechanism: Not confirmed ✗
Practical effect: No effect
Baseline tuning: swept over 3 configs

Limitations:

Only the registered built-in sequence forecasting track was tested. No vision, tabular, dynamics, graph-matching, larger K=32, or deeper multi-layer attention evaluation was run; the implementation used a three-coordinate explicit pair-feature map and a small OT attention layer rather than the full proposed high-dimensional feature basis.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Failed ✗

Agent confidence: 10/10

Implemented and evaluated gauge-free covariance-whitened OT attention on the registered sequence track with the shared transformer_tiny-style architecture and canonical training loop. The idea did not improve test MSE: baseline mean 1.0048756599 versus idea mean 1.0048756525, paired permutation p=1.0. The trained-model mechanism signature failed quantitatively, with observed idea-to-baseline row/column plan-change ratio 6200.737 rather than suppression.

Baseline: map[best_cfg:map[delta:0.0001 lr:0.003] full_mean:1.004875659942627 full_std:0.08302304230055646 per_seed:[1.0032966136932373 0.8694341778755188 1.1456023454666138 1.081549882888794 0.9330782294273376 0.9555575847625732 1.0619580745697021 0.9885283708572388] sweep:[map[cfg:map[delta:0.0001 lr:0.001] mean:1.0469075739383698] map[cfg:map[delta:0.0001 lr:0.003] mean:1.004875659942627] map[cfg:map[delta:0.0001 lr:0.01] mean:1.015213631093502]]]
Comparison: map[delta_mean:-7.450580596923828e-09 idea_wins:2 n_pairs:8 p_value:1 system_worked:false verdict:no measurable effect]
Idea: map[best_cfg:map[delta:0.0001 lr:0.003] mean:1.0048756524920464 per_seed:[1.0032966136932373 0.8694342374801636 1.1456023454666138 1.0815497636795044 0.9330782294273376 0.9555575847625732 1.0619581937789917 0.9885282516479492] std:0.0830230295635864]
Mechanism signature: map[baseline_interaction_plan_rms:1.726159410964101e-07 baseline_rowcol_plan_rms:4.0958703095839155e-10 confirmed:false idea_interaction_plan_rms:5.0314388317929115e-06 idea_rowcol_plan_rms:2.545942379583721e-06 observed_ratio:6200.737450574088 prediction:quotient attention suppresses row/column nuisance sensitivity while retaining interaction sensitivity]

Benchmark result