Add a structured boundary-like operation to an MoE router that rapidly mixes expert probabilities toward a learned distribution while preserving predefined expert groups. The operation is a rank-one stochastic kernel, so it costs linear rather than quadratic work in the number of experts and can act as a controlled reset when routing becomes concentrated.
Treat a spatial feature map or lattice-indexed embedding as a function on a d-dimensional discrete grid and penalize excessive concentration near a chosen anchor using the inverse-radial Hardy weight. Calibrate the penalty with the theorem's high-dimensional scaling 2^ell d^ell instead of selecting an arbitrary spatial L2 coefficient.
Convert an attention or MoE routing affinity matrix into a soft graph and constrain its K_r-density relative to its observed K_s-density. The regularizer penalizes pathological affinity patterns in which moderate s-way coactivation is accompanied by an implausibly low or unstable r-way coactivation.
Augment spatial training examples by replacing a compact active region with several separated components while preserving its exact value histogram, total active area, and amplitude. The augmentation probes the nonlinear interaction between diffusion-like receptive fields and threshold activations, which the paper shows can make fragmented and compact inputs evolve in opposite directions despite identical distributions.