# Orthogonal transitivity projection for pairwise logits

- ID: 188
- Canonical URL: https://synthcore.org/idea/188/orthogonal-transitivity-projection-for-pairwise-logits
- API JSON: https://synthcore.org/api/idea/188.json
- API Markdown: https://synthcore.org/api/idea/188.md
- Verification status: mechanism_works
- Source: [arXiv:2608.25923](https://arxiv.org/abs/2608.25923)
- Category: geometry
- Solves: accuracy, stability, generalization
- ML areas: loss, graph-nn, embedding
- Math tags: linear-algebra, geometry, optimization
- Ratings: usefulness 6/10; difficulty 3/10; novelty 6/10

## Idea description

Given arbitrary pairwise preference logits, project their skew-symmetric part onto the additive-consistent subspace before converting logits into probabilities or rankings. This removes cyclic inconsistency using the Frobenius-nearest consistent matrix, guaranteeing transitive pairwise predictions while preserving the closest possible signal under squared error.

## Mathematical statement

Let S=[s_{ij}] be an n by n skew-symmetric matrix, with Frobenius inner product <X,Y>=sum_{i,j} x_{ij}y_{ij}. The additive-consistent subspace A_n consists of matrices S_{ij}=u_i-u_j. The paper constructs an orthogonal basis B_1,...,B_{n-1} of A_n, so <B_k,B_l>=0 for k not equal to l. Theorem 6.3 gives the nearest Frobenius projection P_A S=sum_k <S,B_k>/<B_k,B_k> B_k. For a positive pairwise-ratio matrix M, Theorem 7.2 applies the same operation in log space: P_C M=exp(P_A(log M)). In the neural adaptation, Z is an unconstrained pairwise-logit tensor, S=(Z-Z^T)/2 is its skew-symmetric component, and S_hat=P_A S is fed to the pairwise loss. The projection is exact, differentiable, and removes every cycle component orthogonal to A_n.

## Key formulas

- $$\mathcal{P}_{\mathcal{A}}S=\sum_{k=1}^{n-1}\frac{\left\langle S,B_k\right\rangle}{\left\langle B_k,B_k\right\rangle}B_k$$
- $$\mathcal{P}_{\mathcal{C}}M=\exp\left(\mathcal{P}_{\mathcal{A}}(\ln M)\right)=\exp\left(\sum_{k=1}^{n-1}\frac{(n-k+1)\left\langle\ln M,B_k\right\rangle}{2n(n-k)}B_k\right)$$
- $$r_{ij}=\exp\left(\sum_{k=1}^{n-1}\frac{(n-k+1)b^{(k)}_{ij}}{2n(n-k)}\sum_{u,v=1}^{n}b^{(k)}_{uv}\ln m_{uv}\right)$$
- $$S=\frac{Z-Z^T}{2},\qquad \widehat S=\sum_{k=1}^{n-1}\frac{\langle S,B_k\rangle}{\langle B_k,B_k\rangle}B_k$$

## Implementation notes

(1) Insert the layer after a network produces pairwise logits Z with shape [batch,n,n], and before Bradley-Terry, sigmoid, softmax, or ranking losses. First form S=(Z-Z.transpose(-1,-2))/2; optionally mask invalid pairs. Precompute basis matrices B[k] with shape [n-1,n,n], using the paper's orthogonal basis or an equivalent QR-orthogonalized basis spanning matrices u 1^T-1 u^T. (2) Compute `denom[k]=(B[k]*B[k]).sum()`, `alpha[k]=(S*B[k]).sum((-1,-2))/denom[k]`, and `S_hat=(alpha[...,k,None,None]*B[k]).sum(-3)`. Use `loss=bce_with_logits(S_hat, pairwise_labels)` and retain gradients through all operations. (3) The basis and projection coefficients are computed exactly from the linear-algebra formula; no spectrum or hyperparameter estimate is required. If ratios M are used instead of logits, compute L=log(clamp(M,eps)), project L, and return exp(L_hat). (4) Start with n=16 or n=32 synthetic Bradley-Terry data: draw true utilities u, label pairs from sigmoid(u_i-u_j), then flip 10%, 20%, and 30% of labels to inject inconsistent cycles. Compare an unconstrained pairwise head, projection only at inference, and projection during training. Report pairwise accuracy, ranking regret, held-out noisy-label loss, number of predicted 3-cycles, and gradient variance. The expected signal is fewer cycles and better noisy-data generalization at similar clean accuracy, with no measurable inference instability.

## Verification

- Status: mechanism_works
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Implemented differentiable Frobenius projection of skew pairwise logits onto the additive-consistent subspace, with numerical checks and matched noisy Bradley–Terry experiments. The math check passed at approximately 1e-15 error; training-time projection eliminated all 3-cycles, improved clean accuracy from 0.814 to 0.904, and reduced ranking regret from 0.101 to 0.027. It increased noisy-label BCE and CPU time, so the observed benefit is consistency and clean-ranking robustness rather than corrupted-label fit or speed.

### Mechanism check

- Verdict: Implemented differentiable Frobenius projection of skew pairwise logits onto the additive-consistent subspace, with numerical checks and matched noisy Bradley–Terry experiments. The math check passed at approximately 1e-15 error; training-time projection eliminated all 3-cycles, improved clean accuracy from 0.814 to 0.904, and reduced ranking regret from 0.101 to 0.027. It increased noisy-label BCE and CPU time, so the observed benefit is consistency and clean-ranking robustness rather than corrupted-label fit or speed.
- Confidence: 8/10
- Limitations: Only n=16 synthetic Bradley–Terry data, two seeds per noise level, and 180 Adam steps were tested. No neural network backbone, held-out pair split, gradient-variance measurement, ratio-space implementation, masking pattern, or larger-scale runtime benchmark was evaluated.

## Artifacts

- [report.md](https://synthcore.org/code/30/report.md)
- [results.json](https://synthcore.org/code/30/results.json)
- [run_experiment.py](https://synthcore.org/code/30/run_experiment.py)
- [Download all files as ZIP](https://synthcore.org/download/30)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.