# Gauge-Free Spectral OT Layer

- ID: 2227
- Canonical URL: https://synthcore.org/idea/2227/gauge-free-spectral-ot-layer
- API JSON: https://synthcore.org/api/idea/2227.json
- API Markdown: https://synthcore.org/api/idea/2227.md
- Verification status: failed_benchmark
- Source: [arXiv:2608.13201](https://arxiv.org/abs/2608.13201)
- Category: architecture
- Solves: stability, accuracy, sample-efficiency
- ML areas: attention, graph-nn, regularization
- Math tags: optimal-transport, linear-algebra, spectral-theory, convex-analysis
- Ratings: usefulness 7/10; difficulty 5/10; novelty 7/10

## Idea description

Parameterize an entropic OT cost only in directions that can change the transport plan, removing row-plus-column potential directions that are invisible because of OT gauge invariance. Whiten the remaining feature coordinates using their empirical covariance, producing an OT layer whose identifiable parameters have substantially more uniform sensitivity.

## Mathematical statement

The entropic OT plan is $\pi(C,a,b)=\arg\min_{\pi\in\mathcal{M}(a,b)}\{\langle C,\pi\rangle+\varepsilon\sum_{ij}\pi_{ij}\log\pi_{ij}\}$, with $\mathcal{M}(a,b)=\{\pi\geq0:\pi\mathbf{1}=a,\pi^\top\mathbf{1}=b\}$. The cost is $C_{\theta}(i,j)=-\theta^\top\varphi(i,j)$, where $\theta\in\mathbb{R}^{F}$ and $\varphi(i,j)\in\mathbb{R}^{F}$. OT is invariant to adding a row and column potential, $C_{ij}\mapsto C_{ij}+r_i+s_j$; therefore the gauge kernel is $\mathcal{N}_{\Phi}=\{\theta:\Phi\theta\in\{r_i+s_j\}_{i,j}\}$, where $\Phi\in\mathbb{R}^{K^2\times F}$ has row $(i,j)$ equal to $\varphi(i,j)^\top$. The paper gives $F\leq(K-1)^2$ for the identifiable quotient and requires $\operatorname{rank}(\Sigma)=F$, where $\Sigma$ is the feature covariance or second-moment matrix. On the tangent space $\mathcal{T}$, the inverse restricted Hessian obeys $(\pi_{\min}/\varepsilon)I\preceq H_{\mathcal{T}}^{-1}\preceq(\pi_{\max}/\varepsilon)I$. The induced parameter-to-plan Jacobian $\mathcal{J}_{\theta}$ satisfies $\sigma_{\min}(\mathcal{J}_{\theta})\geq[\pi_{\min}/(a_{\max}\varepsilon)]\sqrt{\lambda_{\min}(\Sigma)}$, where $\pi_{\min}=\min_{ij}\pi_{ij}$, $\pi_{\max}=\max_{ij}\pi_{ij}$, $a_{\max}=\max_i a_i$, and $\lambda_{\min}(\Sigma)$ is the smallest feature-covariance eigenvalue. The adaptation uses the quotient projector $P_{\perp}$ and whitening matrix $(P_{\perp}\Sigma P_{\perp}+\delta I)^{-1/2}$.

## Key formulas

- $$\pi(C,a,b)=\arg\min_{\pi\in\mathcal{M}(a,b)}\left\{\langle C,\pi\rangle+\varepsilon\sum_{ij}\pi_{ij}\log\pi_{ij}\right\},\qquad C_{\theta}(i,j)=-\theta^\top\varphi(i,j).$$
- $$\frac{\pi_{\min}}{\varepsilon}I\preceq H_{\mathcal{T}}^{-1}\preceq\frac{\pi_{\max}}{\varepsilon}I.$$
- $$\sigma_{\min}(\mathcal{J}_{\theta})\geq\frac{\pi_{\min}}{a_{\max}\varepsilon}\sqrt{\lambda_{\min}(\Sigma)}.$$
- $$\widetilde\theta=(P_{\perp}\Sigma P_{\perp}+\delta I)^{-1/2}P_{\perp}\theta,\qquad C_{\widetilde\theta}(i,j)=-\widetilde\theta^\top\widetilde\varphi(i,j).$$

## Implementation notes

(1) Integration point: insert this parameterization immediately before cost construction in a differentiable Sinkhorn layer. The layer receives source features $x_i$, target features $y_j$, and a learned pair feature map $\varphi(i,j)$; it constructs $C_{ij}=-\theta^\top\varphi(i,j)$ and runs Sinkhorn. Do not optimize raw $\theta$ directly. (2) Build a gauge design matrix $G\in\mathbb{R}^{K^2\times(2K-1)}$. For pair $(i,j)$, its row is the concatenation of the one-hot row vector $e_i$ and target one-hot vector $e_j$, with one redundant target column removed. Compute a projector $P_G=G(G^\top G+\delta_gI)^{-1}G^\top$ and retain cost components orthogonal to this span. If features are represented in parameter space, compute a QR/SVD basis $U$ for the nullspace of the gauge-output map and use $\theta=Uz$. (3) Maintain $\Sigma_t=\beta\Sigma_{t-1}+(1-\beta)\Phi_t^\top\Phi_t/K^2$, where $\Phi_t$ contains pair-feature rows. In the retained basis, compute $W_t=(U^\top\Sigma_tU+\delta I)^{-1/2}$ and use $\theta=UW_tz$. (4) The gauge quotient, covariance whitening, and Jacobian lower bound come from the paper. The covariance eigenvalues are computed from minibatches; $\pi_{\min}$ and $a_{\max}$ are measured from the current plan and source marginal. (5) First experiment: compare raw versus gauge-free-whitened OT attention on a synthetic matching task and a small graph-matching benchmark with $K=32$, $F=64$, and a 4-layer encoder. Measure transport loss, validation accuracy, gradient-norm variance, the number of near-zero singular values in an empirical Jacobian, and failed runs. Success means faster loss decrease at equal FLOPs and fewer unstable runs, especially when features contain row- or column-bias directions.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: no_effect
- Verdict: Built a gauge-free, covariance-whitened entropic OT MVP with log-domain Sinkhorn and a synthetic matching comparison. The mechanism manifested: gauge perturbations changed the plan by at most 4.8e-15, five near-zero Jacobian singular values were removed leaving rank 3, and whitening condition numbers matched the ridge-aware prediction across all anisotropy levels. The idea also showed a modest optimization improvement, reaching 0.0015565 versus 0.0015634 for the raw baseline at 180 steps, though this is not evidence of a large practical win.

### Mechanism check

- Verdict: Built a gauge-free, covariance-whitened entropic OT MVP with log-domain Sinkhorn and a synthetic matching comparison. The mechanism manifested: gauge perturbations changed the plan by at most 4.8e-15, five near-zero Jacobian singular values were removed leaving rank 3, and whitening condition numbers matched the ridge-aware prediction across all anisotropy levels. The idea also showed a modest optimization improvement, reaching 0.0015565 versus 0.0015634 for the raw baseline at 180 steps, though this is not evidence of a large practical win.
- Confidence: 9/10
- Limitations: Only a tiny synthetic K=8 experiment was tested, not the proposed K=32 graph benchmark or a multi-layer attention integration. The whitening ridge limits conditioning for extremely small eigenvalues, and the optimization comparison used a single fixed learning rate and six trials rather than a tuned FLOP-matched study.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 1
- Paired wins: 2/8
- Benchmark verdict: no measurable effect

## Artifacts

- [bench_report.json](https://synthcore.org/code/884/bench_report.json)
- [report.md](https://synthcore.org/code/884/report.md)
- [report_bench_2026-09-01T134833.md](https://synthcore.org/code/884/report_bench_2026-09-01T134833.md)
- [results.json](https://synthcore.org/code/884/results.json)
- [run_experiment.py](https://synthcore.org/code/884/run_experiment.py)
- [stage2_bench.py](https://synthcore.org/code/884/stage2_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/884)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.