Metric-affine asymmetric contrast loss
Implementation & benchmark of arXiv:2608.16714 — A two-point approach to the inverse problem in information geometry
Source paper: A two-point approach to the inverse problem in information geometry arXiv:2608.16714 ⓘ · analyzed Sep 1, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace the symmetric Euclidean contrastive loss between embeddings with a two-point quadratic contrast whose displacement is generated by a local affine connection and measured using the metric at the source endpoint. Because the metric and transport need not be compatible, the loss can be asymmetric, allowing the model to represent directional relations between examples.
Formulas
Mathematical statement
The paper constructs a strongly convex neighborhood from a covering, $\mathscr{U}_{\mathcal C}=\bigcup_{C\in\mathcal C}(C\times C)$, where each pair admits a unique local geodesic interpolation and parallel transport. For an embedding $z\in\mathbb R^d$, use a local metric matrix $G(z)\succ0$ and connection matrices $A_i(z)\in\mathbb R^{d\times d}$. For a pair $(x,y)$ in one local chart, let $v=y-x$ and define parallel transport along $\gamma(t)=x+tv$ by $\frac{dP(t)}{dt}=-\sum_{i=1}^d v_iA_i(\gamma(t))P(t)$, $P(0)=I$. The transported displacement is $u_{x\to y}=P(1)^{-1}v$, and the adapted contrast is $D(x,y)=\frac12u_{x\to y}^{\mathsf T}G(x)u_{x\to y}$. Here $G(x)$ is the source-point metric, $A_i$ are affine-connection matrices, $P(t)$ is transport, and $u_{x\to y}$ is the transported displacement. If $G=I$ and $A_i=0$, this reduces to the ordinary squared Euclidean distance. Since the source metric and transport direction differ between $(x,y)$ and $(y,x)$, generally $D(x,y)\ne D(y,x)$.
Implementation notes
(1) Integrate the method at the embedding-loss stage of a Siamese encoder, supervised-contrastive model, or retrieval model. Let the encoder output $z=f_\theta(s)\in\mathbb R^d$. Attach a small head producing a lower-triangular matrix $L(z)$ with positive diagonal and define $G(z)=L(z)L(z)^{\mathsf T}+\epsilon I$. Initially use globally learned constant connection matrices $A_1,\ldots,A_d$; only later test an MLP that makes them depend on $z$. (2) For every anchor-positive-negative triplet, compute $x=f_\theta(a)$, $y^+=f_\theta(p)$, and $y^-=f_\theta(n)$. For each pair $(x,y)$, set $v=y-x$, initialize $P=I_d$, and use $K$ midpoint steps. At step $k$, evaluate $\gamma_k=x+(k+1/2)v/K$, form $B_k=\sum_i v_iA_i(\gamma_k)$, and update $P\leftarrow(I_d-B_k/K)P$. After the loop solve $Pu=v$ for $u$ rather than explicitly computing an inverse, then evaluate $D(x,y)=u^TG(x)u/2$. Optimize the margin expression in the displayed formula, with $m>0$. (3) The discretized transport ODE and quadratic form come from the mathematical construction; the metric and connection parameters are learned. Estimate numerical stability by monitoring the smallest singular value of every $P$ and clip or spectrally normalize the $A_i$ if it approaches zero. The log-determinant term prevents metric collapse, while the Frobenius penalty controls excessive transport curvature. Use $A_i=0$ and $G=I$ as mandatory ablations. (4) Run a first cheap experiment with a small ResNet or ViT on CIFAR-10 using supervised triplet loss. Compare Euclidean contrastive loss, a learned symmetric Mahalanobis loss, and the metric-affine loss at matched encoder FLOPs and approximately matched parameter counts. Evaluate Recall@1, classification accuracy from a frozen linear probe, and performance when query and gallery examples are subjected to different augmentations or directional domain shifts. Also report the asymmetry gap $|D(x,y)-D(y,x)|$, transport condition numbers, and loss-curve variance. A useful success criterion is a statistically significant retrieval or robustness gain over the Mahalanobis baseline without exploding transport condition numbers or materially slower optimization.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.