arXiv:2607.27116 2026 Regularization 1 ideas extracted · analyzed Aug 31, 2026
The paper establishes a weak-majorization bound for singular values of sums of binary Kronecker products, extending the positive-semidefinite two-term eigenvalue result to arbitrary numbers of arbitrary matrices. For a tensorized neural-network weight matrix, this gives simultaneous certified upper bounds on every Ky Fan norm, not merely the spectral norm. The transferable asset is a cheap factor-level spectral certificate: singular values of the small factors can be computed or approximated and combined through sorted Kronecker products, avoiding an SVD of the large assembled layer. The most direct ML use is a spectral regularizer or constraint for tensorized MLP or attention projections that controls worst-case amplification and the entire singular-value tail.
Parameterize a large linear layer as a sum of binary tensor products, W = Σ_l A_l ⊗ B_l, and regularize a factor-level upper bound on its top-k singular-value sums. The bound controls all Ky Fan norms of W while requiring SVDs only of the small factors, making it suitable for tensorized MLP or attention projections.