# Conditional Sinkhorn Adversarial Augmentation

- ID: 1381
- Canonical URL: https://synthcore.org/idea/1381/conditional-sinkhorn-adversarial-augmentation
- API JSON: https://synthcore.org/api/idea/1381.json
- API Markdown: https://synthcore.org/api/idea/1381.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.24983](https://arxiv.org/abs/2607.24983)
- Category: regularization
- Solves: accuracy, generalization, stability
- ML areas: loss, regularization, gan, diffusion, training-dynamics
- Math tags: optimal-transport, optimization, probability, geometry
- Ratings: usefulness 7/10; difficulty 6/10; novelty 6/10

## Idea description

Replace unconstrained input perturbations or generic distribution shifts with a conditional adversarial generator whose samples remain on a prescribed generator manifold. For each context x, maximize downstream loss over generator parameters within a debiased Sinkhorn-divergence radius of the nominal conditional generator, then minimize predictor loss against the resulting worst-case samples.

## Mathematical statement

The paper's population problem is equation (1.1): w*_{rho,epsilon}(x) is a decision minimizing worst-case expected loss over adversarial generator parameters psi in Psi subject to S_epsilon(Q_psi(·|x), P_hatphi(·|x)) <= rho. Here x is the observed context, w is the neural-network decision or prediction, Y is the generated output, f(w,Y) is the downstream loss, P_hatphi(·|x) is the nominal conditional law generated by parameters hatphi, Q_psi(·|x) is the adversarial conditional law generated by parameters psi, Psi is the allowed adversarial-generator parameter set, rho >= 0 is the ambiguity radius, and S_epsilon is the debiased Sinkhorn divergence with entropic regularization epsilon. The constraint restricts the adversary to structurally admissible generator outputs while the transport divergence measures displacement using samples rather than likelihoods. For empirical sample sets A={a_i}_{i=1}^n and B={b_j}_{j=1}^m, estimate the regularized transport cost OT_epsilon and debias it as S_epsilon(A,B)=OT_epsilon(A,B)-0.5 OT_epsilon(A,A)-0.5 OT_epsilon(B,B). A differentiable penalty enforces the radius using lambda >= 0, followed by projected multiplier ascent.

## Key formulas

- $$w_{\rho,\varepsilon}^{\star}(x)\in\arg\min_{w\in\mathcal{W}}\;\sup_{\psi\in\Psi:\,S_{\varepsilon}(Q_{\psi}(\cdot\mid x),P_{\hat{\phi}}(\cdot\mid x))\leqslant\rho}\mathbb{E}_{Y\sim Q_{\psi}(\cdot\mid x)}\!\left[f(w,Y)\right].$$
- $$S_{\varepsilon}(A,B)=\operatorname{OT}_{\varepsilon}(A,B)-\frac{1}{2}\operatorname{OT}_{\varepsilon}(A,A)-\frac{1}{2}\operatorname{OT}_{\varepsilon}(B,B).$$
- $$\mathcal{L}_{\mathrm{adv}}(w,\psi,\lambda;x)=\frac{1}{n}\sum_{i=1}^{n}f\!\left(w,y_i^{\psi}\right)-\lambda\left[ S_{\varepsilon}(\{y_i^{\psi}\}_{i=1}^{n},\{y_j^{\hat\phi}\}_{j=1}^{n})-\rho\right]_{+},\qquad [z]_{+}=\max(0,z).$$
- $$\psi\leftarrow\psi+\eta_{\psi}\nabla_{\psi}\mathcal{L}_{\mathrm{adv}},\qquad w\leftarrow w-\eta_w\nabla_w\frac{1}{n}\sum_{i=1}^{n}f(w,y_i^{\psi}),\qquad \lambda\leftarrow\left[\lambda+\eta_{\lambda}\left(S_{\varepsilon}-\rho\right)\right]_{+}.$$

## Implementation notes

1. Integration point: use this as a robust-training wrapper around a predictor, policy, or conditional diffusion/GAN generator. Given a context minibatch x, keep a frozen nominal generator G_hatphi(x,z), and add an adversarial copy G_psi(x,z). The predictor receives samples from G_psi during the outer update. A lightweight residual adapter on the nominal generator is preferable to duplicating the entire model. 2. Pseudocode: sample x and noises z_i; compute y_i_adv=G_psi(x,z_i) and y_i_nom=G_hatphi(x,z'_i); compute L_task=mean_i f(w,y_i_adv); construct pairwise ground-cost matrices C_adv,nom, C_adv,adv, and C_nom,nom; run Sinkhorn iterations with entropic coefficient epsilon to obtain OT_epsilon for all three pairs; set S=OT_adv,nom-0.5*OT_adv,adv-0.5*OT_nom,nom; set L_adv=L_task-lambda*relu(S-rho); take K gradient-ascent steps on psi while detaching w; then take one gradient-descent step on w using the final adversarial samples while detaching psi; update lambda=max(0,lambda+eta_lambda*(S-rho)). 3. Compute from the paper: the conditional generator restriction, Sinkhorn-radius constraint, and min-max ordering. Estimate empirically: transport costs, radius violations, and rho. Warm-start psi and choose rho from a quantile of nominal-versus-perturbed Sinkhorn distances. 4. First experiment: train a small conditional trajectory predictor or CIFAR conditional classifier, comparing ordinary augmentation, unconstrained adversarial augmentation, and this method at equal generated samples and FLOPs. Measure clean accuracy, rare-context or corruption loss, Sinkhorn violation rate, and adversarial loss. Success means lower shifted-distribution loss with limited clean-accuracy degradation, S remaining near rho, and stable outer optimization.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: no_effect
- Verdict: Built a differentiable conditional-generator wrapper using a residual adversarial shift, debiased entropic Sinkhorn divergence, radius penalty, projected multiplier ascent, and outer predictor descent. The mechanism checks succeeded: S(delta) fit k*delta^2 with k=0.9858 and maximum relative error 0.276%, the rho=0.36 boundary was predicted at delta=0.6043 and observed at 0.6 (error 0.0043), S(A,A)=0, and lambda stayed zero below rho while increasing after violations. The tiny downstream regression comparison did not produce a win: Sinkhorn training retained clean MSE 0.1893 but had shifted MSE 0.4956 versus ordinary 0.4768; unconstrained augmentation traded clean MSE 0.4538 for shifted MSE 0.1956. Thus worked=true reflects confirmed mathematical mechanism, not an observed application-level improvement.

### Mechanism check

- Verdict: Built a differentiable conditional-generator wrapper using a residual adversarial shift, debiased entropic Sinkhorn divergence, radius penalty, projected multiplier ascent, and outer predictor descent. The mechanism checks succeeded: S(delta) fit k*delta^2 with k=0.9858 and maximum relative error 0.276%, the rho=0.36 boundary was predicted at delta=0.6043 and observed at 0.6 (error 0.0043), S(A,A)=0, and lambda stayed zero below rho while increasing after violations. The tiny downstream regression comparison did not produce a win: Sinkhorn training retained clean MSE 0.1893 but had shifted MSE 0.4956 versus ordinary 0.4768; unconstrained augmentation traded clean MSE 0.4538 for shifted MSE 0.1956. Thus worked=true reflects confirmed mathematical mechanism, not an observed application-level improvement.
- Confidence: 8/10
- Limitations: This is a one-dimensional synthetic conditional-regression toy, not a CIFAR, trajectory, GAN, or diffusion experiment. The adversarial generator is only a scalar residual adapter, the Sinkhorn implementation uses small uniform empirical samples and fixed iterations, there is one seed, and the final multiplier still permits a modest radius violation; no FLOP or wall-clock comparison was performed.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0797
- Paired wins: 6/8
- Benchmark verdict: no measurable effect

## Artifacts

- [bench_experiment.py](https://synthcore.org/code/538/bench_experiment.py)
- [bench_report.json](https://synthcore.org/code/538/bench_report.json)
- [experiment.py](https://synthcore.org/code/538/experiment.py)
- [report.md](https://synthcore.org/code/538/report.md)
- [report_bench_2026-08-31T172039.md](https://synthcore.org/code/538/report_bench_2026-08-31T172039.md)
- [results.json](https://synthcore.org/code/538/results.json)
- [Download all files as ZIP](https://synthcore.org/download/538)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.