Geometry-conditioned integral attention
Implementation & benchmark of arXiv:2607.01128 — GAIA: Geometry-Adaptive Operator Learning for Forward and Inverse Problems
Source paper: GAIA: Geometry-Adaptive Operator Learning for Forward and Inverse Problems arXiv:2607.01128 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace a geometry-agnostic kernel or standard point-cloud attention layer with an integral transform whose weights are conditioned on geometry tokens. The layer receives field values at arbitrary points, domain indicators, boundary features, and local geometric descriptors, and uses cross-attention to produce location-dependent kernels without graph construction.
Formulas
Mathematical statement
The paper formulates a PDE as $\mathcal{D}(c,u)=q$ for $x\in D\subset\mathbb{R}^{d}$ with boundary condition $\mathcal{B}(u)=u_b$ for $x\in\partial D$, where $u:D\to\mathbb{R}^{m}$ is the solution, $c$ is a physical-parameter field, $q$ is forcing, $u_b$ is the boundary value, and $\chi_D$ is the domain indicator. The complete input is $a=(c,q,u_b,\chi_D)\in\mathcal{A}$ and the forward operator is $\mathcal{S}:\mathcal{A}\to\mathcal{U}$, $a\mapsto u$. GAIA's transferable construction is an integral layer with a kernel adapted by geometry tokens. For query points $x_i$, source points $y_j$, field features $v_j$, and geometry-token features $g_k$, use cross-attention scores $s_{ik}=\phi_q(v_i,x_i)^{\mathsf T}\phi_k(g_k)/\sqrt{r}$, where $r$ is the key dimension, and weights $\alpha_{ik}=\exp(s_{ik})/\sum_{\ell}\exp(s_{i\ell})$. The geometry-conditioned kernel is $K_{ij}=\psi(v_i,v_j,x_i,y_j,\sum_k\alpha_{ik}\phi_v(g_k))$, and the discretized integral transform is $z_i=\sigma\left(W_0v_i+\sum_jw_jK_{ij}v_j\right)$, where $w_j$ are quadrature or uniform sampling weights and $\sigma$ is an activation. Conditioning $K_{ij}$ on geometry tokens lets the same network handle changing boundaries, holes, resolutions, and point layouts.
Implementation notes
(1) Integration point: implement this as a drop-in replacement for one or more self-attention or Fourier/operator-kernel blocks in a point-cloud or mesh-free neural operator. Sample source points $y_j$ and query points $x_i$ independently; attach to every point its coordinates, field values, boundary flag, signed-distance estimate, and domain indicator $\chi_D(x)\in\{0,1\}$. Build geometry tokens from boundary points, global shape statistics, and optionally pooled interior features. (2) Pseudocode: encode each source point as $v_j=E_v([u(y_j),c(y_j),q(y_j),x_j,\chi_D(y_j)])$ and each geometry token as $g_k=E_g([x_k,\mathrm{normal}_k,\mathrm{SDF}_k,\mathrm{boundary\_flag}_k])$; for every query $i$, compute $q_i=\phi_q(v_i)$, $k_k=\phi_k(g_k)$, $\alpha_{ik}=\mathrm{softmax}_k(q_i^Tk_k/\sqrt r)$, and $\widetilde g_i=\sum_k\alpha_{ik}\phi_v(g_k)$; then compute $z_i=\sigma(W_0v_i+\sum_jw_j\psi(v_i,v_j,x_i,y_j,\widetilde g_i)v_j)$. Use random point subsampling and uniform weights initially, then test area-aware quadrature weights. (3) Compute from the mathematics: the normalized attention weights and weighted discrete integral are exact implementation objects; estimate geometry features such as signed distance, normals, and quadrature weights from the available point cloud. Do not build a graph. (4) First experiment: train a small 2D Darcy or Poisson surrogate on randomly deformed domains and evaluate on unseen shapes and point resolutions. Compare a standard point transformer, a geometry-blind integral neural operator, and this module at matched parameter count and FLOPs. Success means lower relative $L^2$ error on unseen geometries, less degradation when point resolution changes, and no superlinear graph-construction cost.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.