Spectral-Certified Sinkhorn Optimizer
Implementation & benchmark of arXiv:2608.13201 — Sinkhorn Linearization and the Spectral Proxy: Unifying the Statistical and Algorithmic Theory of Feature-Parameterized Inverse Optimal Transport via a Single Spectral Sandwich
Source paper: Sinkhorn Linearization and the Spectral Proxy: Unifying the Statistical and Algorithmic Theory of Feature-Parameterized Inverse Optimal Transport via a Single Spectral Sandwich arXiv:2608.13201 ⓘ · analyzed Sep 1, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Use the OT spectral bound as a conditioning signal for optimizing parameters of a neural cost or inverse-OT objective. Adapt the parameter step size and add a covariance floor whenever the estimated Jacobian lower bound collapses, preventing optimization from entering regions where Sinkhorn outputs become insensitive to the learned cost.
Formulas
Mathematical statement
Let $Q_{\theta}$ denote the observed conditional transition operator induced by the entropic OT plan, and let $\mathcal{J}_{\theta}=\partial Q_{\theta}/\partial\theta$ be its parameter Jacobian. The paper's core estimate is $\sigma_{\min}(\mathcal{J}_{\theta})\geq\gamma_{\theta}$ with $\gamma_{\theta}=\pi_{\min}\sqrt{\lambda_{\min}(\Sigma)}/(a_{\max}\varepsilon)$. For a cross-entropy objective built from the model operator, the paper derives local strong convexity $\mu\geq\pi_{\min}^{2}\lambda_{\min}(\Sigma)/\varepsilon^{2}=a_{\max}^{2}\gamma_{\theta}^{2}$. Here $\varepsilon>0$ is entropic regularization, $\pi_{\min}$ is the smallest plan entry, $\Sigma$ is feature covariance, $a_{\max}$ is the largest source marginal, and $\lambda_{\min}(\Sigma)$ measures feature identifiability. The paper also gives $L\leq\varepsilon\|\Phi^\top S_a\|_{\mathrm{op}}/[\pi_{\min}\lambda_{\min}(\Sigma)]$, where $\Phi$ is the feature matrix and $S_a$ is the source-marginal weighting operator. The adaptation uses $\gamma_{\theta}$ to scale gradients and uses an empirical Hessian estimate for a safeguarded step size.
Implementation notes
(1) Integration point: modify the optimizer update for trainable cost parameters in a Sinkhorn layer, inverse-OT head, or OT-based attention module. Keep ordinary Adam or SGD for the rest of the network and apply the spectral rule only to $\theta$. (2) For each minibatch, compute the Sinkhorn plan $\pi_t$, pair-feature matrix $\Phi_t$, and covariance EMA $\Sigma_t=\beta\Sigma_{t-1}+(1-\beta)\Phi_t^\top\Phi_t/K^2$. Estimate $\widehat\pi_{\min,t}$ with the 1st percentile of plan entries to avoid a single numerical outlier, set $\widehat a_{\max,t}=\max_i a_i$, and compute $\widehat\lambda_{\min}(\Sigma_t)$ using an eigensolver for small feature dimension or Lanczos for large dimension. (3) Form $\widehat\gamma_t=\widehat\pi_{\min,t}\sqrt{\widehat\lambda_{\min}(\Sigma_t)+\delta}/(\widehat a_{\max,t}\varepsilon)$. Let $g_t$ be the ordinary gradient of the loss with respect to $\theta$. Update $\theta$ with $g_t/\max(\widehat\gamma_t,\gamma_0)$, clip the resulting norm, and optionally multiply by $\widehat\gamma_t/\operatorname{EMA}(\widehat\gamma_t)$ to suppress steps from poorly identifiable batches. (4) Estimate the local smoothness $\widehat L_t$ every $M$ steps with one or two Hessian-vector products and power iteration, then use $\eta_t=\eta_0/\max(\widehat L_t,L_0)$. The paper's formulas determine the conditioning diagnostic; only $\widehat L_t$ is empirically estimated. If $\widehat\gamma_t$ remains small, temporarily increase $\varepsilon$ or add covariance regularization. (5) First experiment: train a small inverse-OT model on synthetic $K=16$ transition matrices with known sparse $\theta$, comparing Adam, Adam plus clipping, and this optimizer. Measure objective versus steps and wall-clock time, parameter error, support recovery, gradient explosions, and correlation between $\widehat\gamma_t$ and subsequent loss decrease. Success means fewer failed runs and equal parameter error in fewer optimizer steps without increasing Sinkhorn iterations.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.