Separable Nonnegative Matrix Factorization Using Powered Ratio-of-Norms Regularization

What the math gives to ML

The paper introduces a powered ratio-of-norms penalty, R_p(X) = ||X||_1^p / ||X||_F, which is a nonconvex sparsity surrogate that distinguishes concentrated representations from diffuse ones more sharply than an ordinary L1 penalty. Its transferable feature is normalization by the Frobenius norm: for p = 1, the penalty is scale-invariant and measures concentration of nonnegative importance mass rather than raw parameter magnitude. A practical neural-network adaptation is to apply this penalty to nonnegative channel, attention-head, or expert importance magnitudes, then prune components with small learned importance.