Unverified
2026
Fractional Ellipsoidal Sparse Attention
Replace dense attention weights with a compactly supported anisotropic bump derived from the obstacle solution, using one learnable ellipsoid per attention head or feature group. Tokens outside the learned ellipsoid receive exactly zero weight, while tokens inside receive smoothly decaying weights according to a fractional exponent. The learned positive-definite matrix represents orientation, scale, and correlations between feature dimensions.
Paper: Ellipsoidal Positivity Sets for Fractional Obstacle Problems with Quadratic Forcing
arXiv:2609.00703