The Rank-Collapse Principle for Quadratic Optimization

What the math gives to ML

The paper isolates a useful consequence of positive-semidefinite low-rank structure: a quadratic score over a complicated feasible family depends only on a low-dimensional shadow, and an optimum generates its own linear exposure direction. This suggests replacing expensive pairwise interaction objectives in discrete neural-network decisions with low-rank quadratic scores whose current solution induces a cheap linear top-k or routing update. The quantitative margin can serve as a stopping or confidence signal, while the paper’s warning is important: low rank alone does not guarantee exact optimization unless the projected feasible geometry is also simple. The most practical transfer is therefore a query-conditioned low-rank quadratic token selector or MoE router, evaluated against ordinary linear top-k routing at equal selection cost.