Stability-calibrated Sinkhorn attention

Integrate this into the attention-logit or MoE-router path before the final weighted aggregation. For a minibatch of n tokens, compute scores Sij = qiᵀkj, define Kij = exp(Sij/ε), set row marginals ri = 1/n and column marginals cj = 1/m, and run 5–20 log-domain Sinkhorn iterations to obtain potentials α and β and transport weights P. Center every potential by subtracting its median to fix the quotient gauge. Maintain an exponential moving average of the observed Sinkhorn residual R = (P1−r, Pᵀ1−c). Estimate the local inverse-stability factor K-hat by finite differences: perturb the centered potentials or logits by small random vectors ξ, recompute the residual, and use K-hat = max ||ξ||∞ / ||ΔR||∞ over several probes, with damping and clipping. Estimate H-hatε from empirical variance or a simple covering proxy of normalized kernel rows; initially use a constant calibrated on a held-out batch. Fit exponents a, p, and r by regressing the envelope, complexity proxy, and K-hat against log ε over a temperature grid. Compute B-hat(ε) from the displayed bound and update ε toward a target noise budget τ. The first experiment should use CIFAR-10 or a small language model, comparing softmax attention, fixed-temperature Sinkhorn attention, and calibrated Sinkhorn attention at equal FLOPs. Measure validation loss, attention-plan variance, entropy, residual size, and loss spikes under small minibatches. Success means lower plan variance and equal-or-better validation loss without more Sinkhorn iterations.