arXiv:2607.06545 2026 Regularization 1 ideas extracted · analyzed Aug 30, 2026
The paper establishes a sharp extremal relationship between higher-order clique densities: fixing the density of K_s gives a minimum possible density of K_r, attained by a one-parameter family of complete multipartite graphons. This provides a computable feasibility envelope linking low-order and high-order co-activation statistics, while the stability result says that near-extremal graphons must be close in cut distance to the corresponding multipartite structure. A practical neural-network transfer is to treat attention or routing affinities as a soft graph and penalize violations of this higher-order density envelope, controlling group-level coactivation without imposing only pairwise sparsity.
Convert an attention or MoE routing affinity matrix into a soft graph and constrain its K_r-density relative to its observed K_s-density. The regularizer penalizes pathological affinity patterns in which moderate s-way coactivation is accompanied by an implausibly low or unstable r-way coactivation.