# Bounded-Influence Hyperbolic Pooling

- ID: 2777
- Canonical URL: https://synthcore.org/idea/2777/bounded-influence-hyperbolic-pooling
- API JSON: https://synthcore.org/api/idea/2777.json
- API Markdown: https://synthcore.org/api/idea/2777.md
- Verification status: unverified
- Source: [arXiv:2608.27876](https://arxiv.org/abs/2608.27876)
- Category: geometry
- Solves: accuracy, generalization, stability
- ML areas: embedding, attention, graph-nn, moe-routing
- Math tags: geometry, differential-geometry, dynamical-systems
- Ratings: usefulness 6/10; difficulty 5/10; novelty 6/10

## Idea description

Replace ordinary token pooling or attention aggregation in a hyperbolic representation space with the point satisfying a bounded radial equilibrium law. Each token contributes a unit tangent direction multiplied by \(\tanh\) of its hyperbolic distance from the candidate, so distant outliers cannot dominate the pooled representation while nearby, geometrically consistent tokens still determine it.

## Mathematical statement

The paper's equilibrium law for Julia's zero is \(\sum_i (\tanh t_i)\mathbf v_i=0\), where \(x_i\) are root points, \(p\) is the candidate point, \(t_i=d(p,x_i)\) is hyperbolic distance, and \(\mathbf v_i\) is the unit tangent vector at \(p\) pointing toward \(x_i\). Its contrasting hyperbolic-zero law is \(\sum_i(\sinh t_i)\mathbf v_i=0\); because \(\tanh t\leq 1\) but \(\sinh t\) grows exponentially, the former has bounded per-token influence and the latter amplifies distant points. We adapt the bounded law to token embeddings \(x_i\) in the hyperboloid model \(\mathbb H^d=\{x\in\mathbb R^{d+1}:\langle x,x\rangle_L=-1,x_0>0\}\), with Lorentz product \(\langle x,y\rangle_L=-x_0y_0+\sum_{k=1}^d x_ky_k\). Define \(t_i=\operatorname{arcosh}(-\langle p,x_i\rangle_L)\), \(u_i=\log_p(x_i)/t_i\) for \(t_i>0\), and solve \(\sum_i\tanh(t_i)u_i=0\). The bounded coefficient supplies the robustness mechanism, while the equilibrium equation is equivariant under hyperbolic isometries.

## Key formulas

- $$\sum_i (\tanh t_i)\,{\bf v}_i=0\qquad\text{and}\qquad\sum_i (\sinh t_i)\,{\bf v}_i=0.$$
- $$\mathbb H^d=\{x\in\mathbb R^{d+1}:\langle x,x\rangle_L=-1,\ x_0>0\},\qquad \langle x,y\rangle_L=-x_0y_0+\sum_{k=1}^{d}x_ky_k.$$
- $$t_i=d(p,x_i)=\operatorname{arcosh}(-\langle p,x_i\rangle_L),\qquad u_i=\frac{\log_p(x_i)}{t_i},\qquad \sum_{i=1}^{m}\tanh(t_i)u_i=0.$$
- $$p_{k+1}=\operatorname{Exp}_{p_k}\!\left(\eta_k\,\frac{1}{m}\sum_{i=1}^{m}\tanh(d(p_k,x_i))\,\frac{\log_{p_k}(x_i)}{\max(d(p_k,x_i),\varepsilon)}\right).$$

## Implementation notes

(1) Integration point: use this operator as a replacement for mean pooling over token embeddings, graph-neighborhood aggregation, or a mixture-of-experts router's summary vector. The encoder outputs points \(x_i\) on the hyperboloid; map an unconstrained vector \(z_i\in\mathbb R^d\) to \(x_i=(\sqrt{1+\|z_i\|^2},z_i)\). Use the pooled point \(p\) as the sequence representation or routing query.

(2) Pseudocode: initialize \(p_0\) as the normalized Lorentz mean or choose \(p_0=x_1\). For \(k=0,\ldots,K-1\), compute \(s_i=-\langle p_k,x_i\rangle_L\), \(t_i=\operatorname{arcosh}(\max(s_i,1+10^{-6}))\), and \(\log_{p_k}(x_i)=\frac{t_i}{\sqrt{s_i^2-1}}(x_i-s_i p_k)\). Set \(g_k=m^{-1}\sum_i\tanh(t_i)\log_{p_k}(x_i)/\max(t_i,\varepsilon)\), then update \(p_{k+1}=\operatorname{Exp}_{p_k}(\eta_k g_k)\), where \(\operatorname{Exp}_p(v)=\cosh(\|v\|_L)p+\sinh(\|v\|_L)v/\|v\|_L\). Stop after 3-8 iterations or when \(\|g_k\|_L<10^{-4}\), and backpropagate through the unrolled iterations.

(3) Computed from the paper's mathematics: the distance, unit tangent, bounded \(\tanh\) influence, and equilibrium update are exact. No spectral or distributional quantities need to be estimated. Empirically compare \(\tanh(t_i)\) with \(t_i\), \(\sinh(t_i)\), and constant weights; clip \(\eta_k\|g_k\|_L\) below 1.0 for numerical stability.

(4) First cheap experiment: train a small hyperbolic graph neural network or hyperbolic transformer on Cora or a synthetic tree-classification benchmark. Compare Euclidean mean pooling, standard hyperbolic Frechet-mean pooling, hyperbolic attention, and this operator using identical encoders and FLOPs. Add controlled corruption by moving 10-40% of token or node embeddings radially outward by factors of 2-10. The predicted signal is nearly unchanged accuracy under distant-minority corruption, lower sensitivity of the pooled point's distance to corrupted tokens, and improved validation accuracy on hierarchical data. Measure iteration overhead and test whether four iterations suffice; success requires robustness at comparable clean-data accuracy and less degradation than \(\sinh\)-weighted aggregation.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.