Inverse-Square Fractional Attention

Integrate this at the attention-logit stage of a vision transformer, point-cloud transformer, or coordinate-aware graph neural network whose tokens have coordinates x_i in R^d. Let c be the singularity center; initially use the image center, point-cloud centroid, or a fixed physical origin. For each head, compute ordinary queries q_i, keys k_j, and values v_j. In parallel compute r_ij=||x_i-x_j||_2+epsilon, u_i=||x_i-c||_2+epsilon, and the kernel K_ij from the displayed formula. Add lambda log(K_ij+epsilon) to the standard scaled dot-product logits, mask invalid pairs, and apply softmax. Normalize coordinates to a bounded domain and use epsilon=10^-4. Initialize lambda=0 to isolate the effect, then learn a nonnegative lambda per head. Optionally learn sigma using sigma=(d-2)/2 sigmoid(b), and parameterize s=(d-2sigma) sigmoid(r) so the paper's admissibility condition is always satisfied. For large token sets, use the k nearest neighbors together with a few radial bins rather than dense pairwise attention. The paper determines the kernel form and admissible exponent range; c, lambda, and optional exponents are estimated by backpropagation. First test on a 32x32 synthetic coordinate-regression task with clipped singular targets such as f(x)=||x-c||^{-gamma}, followed by a point-cloud classification benchmark. Compare vanilla attention and relative-position bias at equal parameters and FLOPs. Success is lower error on singular or multiscale targets, faster loss descent, or better accuracy under token subsampling. Permuting coordinates is a negative control and should remove any gain.