△ Mechanism confirmed, baseline not beaten 2026

Conditional OT barycenter feature augmentation

Implementation & benchmark of arXiv:2607.19761 — Harnessing Heterogeneous Data for Conditional Optimization via Optimal Transport

Usefulness7/10
Difficulty5/10
Novelty4/10

Source paper: Harnessing Heterogeneous Data for Conditional Optimization via Optimal Transport arXiv:2607.19761 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Construct synthetic latent examples from an optimal-transport barycenter of several source domains, restricting the barycentric mass to the context region relevant to the prediction. The resulting representations preserve cross-source consensus while reducing domain-specific nuisance variation. Train on the original examples plus barycentric latent examples with transported soft labels.

Formulas

$$Q_{\mathrm{bar}}\in\arg\min_Q\sum_{k=1}^{K}\theta_kW(Q,P_k),\qquad P_k=\sum_{j=1}^{N_k}w_{k,j}\delta_{\widehat{\boldsymbol{\xi}}_{k,j}}$$
$$\mathcal{V}_{\boldsymbol{\alpha}}=\operatorname{conv}\left(\left\{\left(c(\boldsymbol{\xi},\widehat{\boldsymbol{\xi}}_{1,\alpha_1}),\ldots,c(\boldsymbol{\xi},\widehat{\boldsymbol{\xi}}_{K,\alpha_K})\right):\boldsymbol{\xi}\in\mathcal{N}^{c}\right\}\right)$$
$$\widetilde{z}_{\boldsymbol{\alpha}}=\sum_{k=1}^{K}u_kz_{k,\alpha_k},\qquad u_k\geq0,\qquad\sum_{k=1}^{K}u_k=1$$
$$\mathcal{L}_{\mathrm{mix}}=\mathcal{L}_{\mathrm{ERM}}+\eta\,\mathbb{E}_{\widetilde{z}\sim Q_{\mathrm{bar}}}\left[g(x)\,\ell_\theta(\widetilde{x},\widetilde{y})\right]$$

Mathematical statement

For source empirical laws $P_k=\sum_jw_{k,j}\delta_{\widehat{\boldsymbol{\xi}}_{k,j}}$, the OT barycenter is $Q_{\mathrm{bar}}\in\arg\min_Q\sum_{k=1}^{K}\theta_kW(Q,P_k)$, where $\theta_k\geq0$, $\sum_k\theta_k=1$, and $W$ is Wasserstein distance. The paper couples one sample from each source using an index tuple $\boldsymbol{\alpha}=(\alpha_1,\ldots,\alpha_K)$ and defines $\mathcal{V}_{\boldsymbol{\alpha}}=\operatorname{conv}(\{(c(\boldsymbol{\xi},\widehat{\boldsymbol{\xi}}_{1,\alpha_1}),\ldots,c(\boldsymbol{\xi},\widehat{\boldsymbol{\xi}}_{K,\alpha_K})):\boldsymbol{\xi}\in\mathcal{N}^{c}\})$. Here $c$ is the transport cost and $\mathcal{N}^{c}$ is the complement of the conditioning region. This convex set means that simultaneous source costs can be combined through nonnegative mixture coefficients. Adapt the barycenter to latent representations $z_{k,i}=h_\phi(x_{k,i})$ and condition its weights using a gate $g(x)$, so the synthetic latent point is a weighted consensus of source features rather than an arbitrary interpolation.

Implementation notes

Integrate the module between the encoder $h_\phi$ and the prediction head. At every step, draw an equal-size minibatch from each of $K$ source domains and compute latent vectors $z_{k,i}=h_\phi(x_{k,i})$. Compute a gate $g_{k,i}$ for the desired context and replace source masses by $w_{k,i}\leftarrow g_{k,i}+\delta$, followed by normalization. Maintain $m$ barycenter support vectors $b_r$ initialized from one source. For each source, calculate squared latent costs $C_{k,ir}=\|z_{k,i}-b_r\|_2^2$ and run $T=20$ Sinkhorn iterations to obtain transport plans $\pi_k$ between source weights and barycenter weights $q_r$. Update the barycenter by $b_r\leftarrow[\sum_k\theta_k\sum_i\pi_{k,ir}z_{k,i}]/[\sum_k\theta_k\sum_i\pi_{k,ir}+\delta]$, then recompute plans for five alternating barycenter iterations. Construct a soft label for each support point as $\widetilde{y}_r=[\sum_k\theta_k\sum_i\pi_{k,ir}y_{k,i}]/[\sum_k\theta_k\sum_i\pi_{k,ir}+\delta]$. Add $\eta$ times the prediction loss on $(b_r,\widetilde{y}_r)$ to the ordinary source loss. Use stop-gradient through $b_r$ and $\pi_k$ initially to avoid unstable encoder-barycenter feedback; later test end-to-end gradients. The exact mathematical component is the weighted barycenter objective and convex combination of source features; the latent cost, number of support points, and Sinkhorn approximation are engineering choices. First test on PACS or Office-Home with three source domains and one held-out domain, comparing ERM, standard Mixup, domain-adversarial training, and barycentric augmentation. Measure held-out accuracy, rare-context accuracy, domain-classifier accuracy on latent features, and added compute. Success is improved held-out accuracy and reduced domain separability with less than 20 percent training overhead; failure is degraded source accuracy or no gain over Mixup.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 7/10

Built a readable latent conditional-OT barycenter augmenter using Sinkhorn transport, gated source masses, transported soft labels, and a detached barycenter loss. The three toy predictions were confirmed: convex-hull violation was 0, consensus variance matched sigma^2/K within 0.99%, and gate-weight movement had slope -2.0 as predicted. However, augmentation did not improve held-out accuracy in this toy task: ERM and Mixup reached 0.924 versus 0.918 for OT, so there is no evidence of a practical accuracy win here.

Agent confidence
7/10
Baseline
ERM: target accuracy 0.9240, source accuracy 0.9479
Idea
Conditional latent OT: target accuracy 0.9180, source accuracy 0.9375; mechanism checks passed with maximum variance relative error 0.00986, convex-hull violation 0, and gate slope -2.0

Limitations: Only a small synthetic domain-shift classification task was tested, not PACS or Office-Home. The variance and gate-linearity checks use idealized consensus constructions rather than estimating those properties from the full entropic OT solver; end-to-end gradients, domain separability, rare-context accuracy, runtime overhead, CUDA execution, and multi-seed significance were not tested.

How to run: python3 ot_barycenter_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.19761)

Ahead of the baseline (−6.5%) but not statistically significant (p=0.3907, wins 5 / 8 paired seeds).

Benchmark
Conditional_ot_domains
Model
shared_mlp_tiny
Paired seeds
8
Baseline mean
0.0775
Idea mean
0.0725
Effect (Δ)
-0.005 (−6.5%; negative = idea better)
Wins
5 / 8 paired seeds
p-value
0.3907 (permutation test, 20 000 shuffles)
Smallest detectable effect
±15.3%
Mechanism
Confirmed ✓
Practical effect
Inconclusive
Baseline tuning
swept over 0 configs
Limitations:

The custom track is a small synthetic domain-generalization benchmark, not PACS or Office-Home. Compute overhead and domain-classifier accuracy were not measured separately. The mechanism signature uses a variance-reduction proxy from trained latent features rather than directly estimating a full barycenter covariance; no end-to-end-versus-detached ablation was run.

How to run: python3 stage2_ot_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark latest Failed ✗

Agent confidence: 9/10

Built conditional OT barycenter feature augmentation with gated source masses, Sinkhorn transport, detached latent barycenters, and transported soft labels. On the structurally matched custom three-source domain-generalization track, the best OT system had lower test error than the tuned ERM baseline, but the paired result was not significant (p=0.39075), so this is not a demonstrated win. The mechanism signature was confirmed at neural-network scale: observed consensus variance was approximately 0.30 of pooled source feature variance, close to the predicted one-third.

Baseline
ERM shared_mlp_tiny; best sweep cfg lr=0.006, baseline full mean error=0.0775000, std=0.0108253, per-seed=[0.0775,0.0725,0.0750,0.0650,0.0825,0.0625,0.0875,0.0975]
Idea
Conditional OT; best cfg lr=0.012, eta=0.15, mean error=0.0725000, std=0.0124373, per-seed=[0.0850,0.0675,0.0825,0.0525,0.0550,0.0775,0.0725,0.0875]
Mechanism signature
predicted consensus variance=0.2850805; observed trained-model proxy=0.2627102; observed reduction ratio=0.3000; confirmed=true
Paired delta
-0.0050000 error, 5/8 paired wins, permutation p=0.39075, verdict=no significant win

Benchmark result

Ahead of the baseline (−6.5%) but not statistically significant (p=0.3907, wins 5 / 8 paired seeds).

Benchmark
Conditional_ot_domains
Model
shared_mlp_tiny
Paired seeds
8
Baseline mean
0.0775
Idea mean
0.0725
Effect (Δ)
-0.005 (−6.5%; negative = idea better)
Wins
5 / 8 paired seeds
p-value
0.3907 (permutation test, 20 000 shuffles)
Smallest detectable effect
±15.3%
Mechanism
Confirmed ✓
Practical effect
Inconclusive
Baseline tuning
swept over 0 configs
Limitations:

The custom track is a small synthetic domain-generalization benchmark, not PACS or Office-Home. Compute overhead and domain-classifier accuracy were not measured separately. The mechanism signature uses a variance-reduction proxy from trained latent features rather than directly estimating a full barycenter covariance; no end-to-end-versus-detached ablation was run.

How to run: python3 stage2_ot_bench.py

Artifacts

Implementation overview ⬇ Download all as ZIP 7 files · code, reports and structured results