Nonnegative Quadratics over a Quadrant with a Bilinear Constraint
arXiv:2608.16836
2026
Architecture
2 ideas extracted · analyzed Sep 1, 2026
What the math gives to ML
The paper gives an explicit semidefinite convexification for quadratic functions on the noncompact region where two nonnegative variables satisfy a bounded product. The transferable asset is the construction of a small lifted moment representation with auxiliary shadow moments and PSD localizing matrices, converting a nonconvex multiplicative relation into tractable matrix constraints. In neural networks this can support differentiable, certified multiplicative gates or structured regularization of pairwise feature interactions, especially in small bottleneck modules where semidefinite penalties are affordable.
Ideas from this paper
Unverified
2026
Replace an unconstrained multiplicative interaction between two nonnegative neural features by a lifted gate whose first and second moments satisfy the paper's semidefinite relaxation for the set F = {(x1,x2): x1,x2 >= 0, x1 x2 <= 1}. Insert the gate into an MLP, attention score, or MoE router to prevent explosive feature products while retaining a tractable convex feasible set.
Useful6/10
Difficulty7/10
Novelty8/10
Unverified
2026
Use the lifted convex hull as a training-time regularizer for pairs of nonnegative neural features, encouraging their empirical second- and third-order interaction statistics to lie in the paper's moment cone. This constrains correlations, squares, and cubic cross-moments jointly through PSD inequalities instead of merely penalizing large activations.
Useful5/10
Difficulty4/10
Novelty8/10