✓✓ Beats tuned baseline 2026

Displacement-Huber distribution pooling

Implementation & benchmark of arXiv:2608.13131 — Huber-Wasserstein barycenters for robust distribution-valued data

Usefulness7/10

Difficulty4/10

Novelty7/10

Architecture Stability Generalization Attention Embedding Graph nn Moe routing Optimal transport Robust statistics Convex analysis

Source paper: Huber-Wasserstein barycenters for robust distribution-valued data arXiv:2608.13131 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace ordinary Wasserstein or arithmetic pooling of distribution-valued features with a barycenter whose individual quantile displacements are Huberized. Small changes between input distributions remain averaged quadratically, while a corrupted token, expert, graph neighborhood, or augmentation cannot move the pooled distribution arbitrarily far. The module is especially cheap for one-dimensional distributions represented by fixed quantile vectors.

Formulas

$$h_\delta(r)=\begin{cases}\frac{1}{2}r^2,&|r|\leq\delta,\\ \delta\left(|r|-\frac{\delta}{2}\right),&|r|>\delta,\end{cases}\qquad \psi_\delta(r)=h_\delta'(r)=\operatorname{clip}(r,-\delta,\delta).$$

$$Q_\star=\arg\min_{Q\ \mathrm{nondecreasing}}\sum_{i=1}^{n}a_i\int_0^1h_\delta\!\left(Q(u)-Q_i(u)\right)\,du,$$

$$\sum_{i=1}^{n}a_i\,\psi_\delta\!\left(Q_\star(u)-Q_i(u)\right)=0\quad\text{for every }u\in(0,1).$$

$$q\leftarrow q-\frac{\sum_i a_i\psi_\delta(q-Q_i)}{\sum_i a_i\mathbf{1}\{|q-Q_i|<\delta\}+\varepsilon},\qquad Q_\star[:,j]=q\text{ for quantile index }j.$

Mathematical statement

For a scalar displacement r and threshold delta>0, use the Huber penalty h_delta(r)=r^2/2 when |r|<=delta and delta(|r|-delta/2) when |r|>delta. Given input probability measures mu_i with one-dimensional quantile functions Q_i(u), u in (0,1), and nonnegative normalized weights a_i with sum_i a_i=1, define the displacement-wise Huber-Wasserstein barycenter by minimizing F(Q)=sum_i a_i integral_0^1 h_delta(Q(u)-Q_i(u))du over nondecreasing quantile functions Q. In one dimension the minimization separates pointwise: for every u, Q_star(u) minimizes sum_i a_i h_delta(q-Q_i(u)). Its first-order condition is sum_i a_i psi_delta(q-Q_i(u))=0, where psi_delta(r)=clip(r,-delta,delta) is the derivative of h_delta. The bounded score satisfies |psi_delta(r)|<=delta, so any one input has limited influence. The convex objective retains quadratic curvature for inlier displacements and linear growth for outliers.

Implementation notes

Integrate this as a pooling layer for a set of distribution-valued features. Represent each input object as a tensor Q of shape [batch, n_objects, m_quantiles], where Q[i,j] is the inverse CDF evaluated at fixed levels u_j=(j+0.5)/m. Obtain object weights a_i from uniform averaging, a supplied mask, or a learned attention/router score followed by softmax normalization. For each batch and quantile index j, initialize q_j with the weighted arithmetic mean of Q[:, :, j]. Run 5 Newton iterations using the stationarity equation: r_i=q_j-Q_i,j, psi_i=clamp(r_i,-delta,delta), active_i=1[abs(r_i)<delta], then q_j -= sum_i(a_i*psi_i)/(sum_i(a_i*active_i)+1e-6). Compute all quantile coordinates in parallel. If upstream features are not guaranteed monotone, sort the resulting quantile vector or parameterize input distributions through positive increments. Estimate delta from a robust scale, delta=c*1.4826*median(abs(Q_i,j-median_i(Q[:, :, j]))) with c in [1,2], or learn one positive delta per layer. The exact mathematical computation is the pointwise convex Huber barycenter; the empirical choices are delta, Newton iteration count, and weight parameterization. First test on CIFAR-100 or a small text classification model where each token feature is converted into a 32-bin soft histogram. Compare mean pooling, ordinary Wasserstein barycenter pooling, and this layer under 20-50% corrupted tokens. Measure clean and corrupted accuracy, gradient norms, and wall-clock overhead. Success means smaller accuracy degradation under corruption with less than 2x pooling cost and no more than a one-point clean-accuracy loss.

Verification

Beats tuned baseline

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Significant win ✓

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built a NumPy displacement-Huber quantile pooling module with a safeguarded Newton/bisection solver and numerical checks. All three mechanism predictions were confirmed: the [0,0,T] saturation transition occurred at T=1.5 as predicted, maximum curve error was 5.6e-17, inlier averaging error was 0, and outlier score influence was capped at 1/3. In the toy classifier, Huber pooling did not improve corruption accuracy (47.75% vs 47.875% for mean pooling), while costing about 30.5x more, so the mechanism works but this MVP showed no practical task-level win.

Agent confidence: 9/10

Baseline: Arithmetic pooling: clean accuracy 1.0000, 50% corruption accuracy 0.47875, degradation 0.52125, 0.03035 ms/sample
Idea: Displacement-Huber pooling: clean accuracy 1.0000, 50% corruption accuracy 0.47750, degradation 0.52250, 0.92541 ms/sample, 30.49x arithmetic-pooling overhead; mechanism maximum saturation error 5.55e-17

Limitations: Only a synthetic nearest-centroid classification task was tested; no CIFAR/text model, learned attention weights, gradient norms, GPU implementation, or ordinary Wasserstein baseline was evaluated. The safeguarded solver uses 12-20 iterations rather than the proposed fixed five Newton steps because the raw update diverges when all residuals are in the Huber linear region.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.13131) Sep 2, 2026

Beats the tuned baseline by 71.8% (p=0.0081, wins 8 / 8 paired seeds; metric: lower is better).

Benchmark: Distribution_token_sequence
Model: custom_poolnet
Paired seeds: 8
Baseline mean: 0.0667
Idea mean: 0.0188
Effect (Δ): -0.0479 (−71.8%; negative = idea better)
Wins: 8 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±16.0%
Mechanism: Confirmed ✓
Practical effect: Helps
Baseline tuning: swept over 9 configs

Limitations:

The custom track is synthetic rather than CIFAR-10 or a natural-language benchmark, and no wall-clock or parameter-count comparison was recorded. The custom task uses fixed uniform token weights and does not test learned attention/router weights, ordinary Wasserstein pooling, or corruption-rate sweeps beyond the fixed 30% contamination setting.

How to run: python3 hub_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Worked ✓

Agent confidence: 8/10

Implemented displacement-Huber quantile pooling as an end-to-end sequence model intervention on a custom distribution-token sequence track, with arithmetic pooling as the matched baseline. Across 8 paired seeds, Huber pooling reduced test MSE from 0.0667215 to 0.0187981, with paired delta -0.0479234 and permutation p=0.0081; all 8 seeds improved. The trained-model mechanism signature was confirmed: a gross token shifted Huber pooling by 0.072727 versus 0.833333 for arithmetic pooling, close to the predicted bounded influence scale 0.066667.

Baseline: test MSE mean=0.0667214682, std=0.0112858199, best lr=0.006, epochs=12
Comparison: delta_mean=-0.0479233962, idea_wins=8/8, permutation_p=0.0081, verdict=idea better (significant)
Idea: test MSE mean=0.0187980720, std=0.0037972448, best lr=0.006, epochs=12, delta=0.8
Mechanism signature: predicted shift scale=0.0666667; observed trained Huber shift=0.0727273; observed trained arithmetic shift=0.8333334; confirmed=true

Benchmark result