Automatic Model-Order Selection for Nonnegative Matrix Factorization via Column $\ell_{2,0}$ Regularization

What the math gives to ML

The paper turns rank selection into optimization over an oversized factorization by penalizing entire paired columns with an exact column \(\ell_{2,0}\) penalty, rather than training separate models at different ranks. The transferable asset is a warm-started regularization path: increasing \(\lambda\) progressively deletes redundant latent directions while preserving the tensor dimensions and optimizer state. Its scale-balancing operation removes the reciprocal-scaling ambiguity of factorized layers, making column norms meaningful for pruning and stabilizing the selection path. A direct neural-network experiment is to replace a dense or LoRA weight update by an initially overparameterized low-rank factorization, apply paired hard-threshold proximal updates, and select the smallest active rank meeting a validation-loss tolerance.