Holder-Stable Density Kernel Attention

Integrate this as a replacement for a small-head attention or graph message-passing kernel, especially when the routing density changes between examples, timesteps, or augmentation views. Given token features h in R^{m x d}, compute a probability density p = softmax(g(h)). Produce a bounded kernel matrix K using K_ij = tanh(q_i^T k_j / sqrt(d)) or K_ij = clip(q_i^T k_j / sqrt(d), -c, c), ensuring that every entry has magnitude at most K_max. Apply the discrete operator with the sequence: s = sqrt(p + epsilon); u = s[:, None] times h; v = K times u; out = s[:, None] times v, optionally followed by a learned value projection and residual connection. The paper's mathematical estimate is the square-root L1-to-Hilbert-Schmidt stability bound. In discrete form, estimate ||p - p'||_1 between two perturbed inputs and verify that ||D_sqrt_p K D_sqrt_p - D_sqrt_p' K D_sqrt_p'||_F scales approximately as its square root. The bound assumes normalized densities and bounded K; temperature, clipping, and residual mixing must be tuned empirically. First experiment on a 2-layer CIFAR-100 ViT or WikiText-2 Transformer with token masking and additive embedding noise, comparing standard softmax attention with this symmetric density-weighted kernel at matched FLOPs. Success is reduced output sensitivity and validation-loss variance under perturbations, with no more than a small clean-accuracy penalty; a log-log slope near one half is the falsifiable stability signal.