Metric-magnitude pooling
Implementation & benchmark of arXiv:2607.23354 — Scalably computing metric magnitude
Source paper: Scalably computing metric magnitude arXiv:2607.23354 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace mean or max pooling over a set of learned element embeddings with pooling based on the metric-magnitude weighting. Pairwise distances create a globally coupled correction for redundancy, so geometrically isolated or boundary elements can contribute differently from dense clusters of nearly duplicate elements.
Formulas
Mathematical statement
For embeddings x_1, ..., x_n in R^m, define the Euclidean metric d_ij = ||x_i - x_j||_2. For a scale t > 0, define the similarity matrix Z by Z_ij = exp(-t d_ij), applied entrywise. A weighting is a vector w satisfying Z w = 1, where 1 is the n-dimensional all-ones vector. The paper defines magnitude as Mag(Z) = 1^T w. For Euclidean metrics, Z is positive definite and the weighting is unique, so w = Z^{-1}1. The neural adaptation pools embeddings as h = (1^T w + epsilon)^{-1} sum_i w_i x_i. Since exact solutions can be poorly conditioned or can contain signed coefficients, use the ridge system (Z + lambda I)w_lambda = 1, monitor 1^T w_lambda, and compare against a nonnegative softmax version q = softmax(beta w_lambda). The transferable property is global redundancy correction: every coefficient depends on all pairwise distances rather than on an independently computed local score.
Implementation notes
Integrate the method at the readout of a set encoder, point-cloud network, graph neural network, or token compressor. Let the upstream network produce X with shape B by n by m. For each batch element, compute the pairwise distance matrix D using squared-distance matrix multiplication followed by a stabilized square root: D_ij = sqrt(max(||x_i||^2 + ||x_j||^2 - 2 x_i^T x_j, 0)). Estimate t from the off-diagonal median distance using t = log(2) divided by median(D) plus 1e-6. Form Z = exp(-tD), add lambda times the identity, and solve the symmetric positive-definite linear system (Z + lambda I)w = ones with a batched Cholesky solver. Compute h = sum_i w_i x_i divided by sum_i w_i plus 1e-6, then pass h to the prediction head. Use the differentiable linear-solve primitive so gradients reach the embeddings. Implement a second mode q = softmax(beta w) and h = sum_i q_i x_i; this avoids unstable signed or nearly cancelling weights. The paper directly determines the kernel, linear system, and magnitude; the engineer estimates t, selects lambda, and tests conditioning empirically. Start with ModelNet40 and a PointNet-style per-point MLP, using 64 to 256 points. Compare mean, max, standard attention, signed metric-magnitude, and softmax metric-magnitude pooling at matched widths. Measure classification accuracy, accuracy after deleting 20 percent of points, loss at equal training steps, and throughput. A successful result is improved redundant-point robustness or accuracy at comparable compute; record the extra O(n squared m + n cubed) cost and restrict the first exact experiment to n <= 256.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.