# Displacement-Huber distribution pooling - ID: 2223 - Canonical URL: https://synthcore.org/idea/2223/displacement-huber-distribution-pooling - API JSON: https://synthcore.org/api/idea/2223.json - API Markdown: https://synthcore.org/api/idea/2223.md - Verification status: beats_baseline - Source: [arXiv:2608.13131](https://arxiv.org/abs/2608.13131) - Category: architecture - Solves: stability, generalization - ML areas: attention, embedding, graph-nn, moe-routing - Math tags: optimal-transport, robust-statistics, convex-analysis - Ratings: usefulness 7/10; difficulty 4/10; novelty 7/10 ## Idea description Replace ordinary Wasserstein or arithmetic pooling of distribution-valued features with a barycenter whose individual quantile displacements are Huberized. Small changes between input distributions remain averaged quadratically, while a corrupted token, expert, graph neighborhood, or augmentation cannot move the pooled distribution arbitrarily far. The module is especially cheap for one-dimensional distributions represented by fixed quantile vectors. ## Mathematical statement For a scalar displacement r and threshold delta>0, use the Huber penalty h_delta(r)=r^2/2 when |r|<=delta and delta(|r|-delta/2) when |r|>delta. Given input probability measures mu_i with one-dimensional quantile functions Q_i(u), u in (0,1), and nonnegative normalized weights a_i with sum_i a_i=1, define the displacement-wise Huber-Wasserstein barycenter by minimizing F(Q)=sum_i a_i integral_0^1 h_delta(Q(u)-Q_i(u))du over nondecreasing quantile functions Q. In one dimension the minimization separates pointwise: for every u, Q_star(u) minimizes sum_i a_i h_delta(q-Q_i(u)). Its first-order condition is sum_i a_i psi_delta(q-Q_i(u))=0, where psi_delta(r)=clip(r,-delta,delta) is the derivative of h_delta. The bounded score satisfies |psi_delta(r)|<=delta, so any one input has limited influence. The convex objective retains quadratic curvature for inlier displacements and linear growth for outliers. ## Key formulas - $$h_\delta(r)=\begin{cases}\frac{1}{2}r^2,&|r|\leq\delta,\\ \delta\left(|r|-\frac{\delta}{2}\right),&|r|>\delta,\end{cases}\qquad \psi_\delta(r)=h_\delta'(r)=\operatorname{clip}(r,-\delta,\delta).$$ - $$Q_\star=\arg\min_{Q\ \mathrm{nondecreasing}}\sum_{i=1}^{n}a_i\int_0^1h_\delta\!\left(Q(u)-Q_i(u)\right)\,du,$$ - $$\sum_{i=1}^{n}a_i\,\psi_\delta\!\left(Q_\star(u)-Q_i(u)\right)=0\quad\text{for every }u\in(0,1).$$ - $$q\leftarrow q-\frac{\sum_i a_i\psi_\delta(q-Q_i)}{\sum_i a_i\mathbf{1}\{|q-Q_i|<\delta\}+\varepsilon},\qquad Q_\star[:,j]=q\text{ for quantile index }j.$ ## Implementation notes Integrate this as a pooling layer for a set of distribution-valued features. Represent each input object as a tensor Q of shape [batch, n_objects, m_quantiles], where Q[i,j] is the inverse CDF evaluated at fixed levels u_j=(j+0.5)/m. Obtain object weights a_i from uniform averaging, a supplied mask, or a learned attention/router score followed by softmax normalization. For each batch and quantile index j, initialize q_j with the weighted arithmetic mean of Q[:, :, j]. Run 5 Newton iterations using the stationarity equation: r_i=q_j-Q_i,j, psi_i=clamp(r_i,-delta,delta), active_i=1[abs(r_i)