M-convex interaction pruning

Apply this to a tensorized MLP or polynomial feature layer. Represent an interaction index by alpha=(alpha_1,...,alpha_n) with nonnegative integer entries summing to degree d. The dense candidate layer computes phi_alpha(h), such as products of projected features, and has one parameter w_alpha per alpha. Begin with a support S containing the largest-magnitude or largest-gradient interactions. Every pruning round scores active interactions using running |w_alpha|, |w_alpha times gradient_alpha|, or a validation sensitivity estimate. Remove the lowest-score alpha and propose a replacement alpha_new from the inactive set. Repair the support using the exchange rule: for each pair alpha,beta in the proposed support and each i with alpha_i>beta_i, search for j with alpha_j<beta_j; if alpha-e_i+e_j or beta+e_i-e_j is absent, insert the missing feasible neighbor with the highest score, or reject the removal if no neighbor exists. After repair, the forward computation is f_S(h)=sum_{alpha in S} w_alpha phi_alpha(h)/alpha!, with the mask fixed between pruning rounds. Directly check the exchange axiom on small supports; for larger supports, check only sampled pairs and maintain a bounded exchange-neighborhood cache. Start with n=16,d=3 and retain 10%, 25%, and 50% of interactions in a CIFAR-100 tensorized MLP or a small Transformer feed-forward block. Compare against magnitude pruning, random pruning, and unstructured sparse training at equal parameter count and retraining budget. Measure validation accuracy or perplexity, activation outliers, gradient variance, and actual wall-clock memory. The target signal is equal accuracy at lower parameter count, smoother loss immediately after pruning, and improved hardware sparsity; a 2x parameter reduction at iso-accuracy is a meaningful positive result.