Computing Strong Rank-Revealing Factorizations for Matrices with Orthonormal Rows
arXiv:2607.13532
2026
Memory
1 ideas extracted · analyzed Aug 30, 2026
What the math gives to ML
The paper develops a mathematically principled way to select a small subset of columns that simultaneously preserves the span of a matrix and keeps the selected basis well-conditioned. Its transferable asset is not ordinary low-rank approximation, but rank-revealing subset selection: pivoting the leading right-singular-vector matrix identifies actual data items or features that serve as a stable basis rather than replacing them with synthetic singular vectors. This suggests representative-token selection for KV-cache compression and activation-dictionary construction, with projection residual and smallest selected singular value providing direct, falsifiable quality diagnostics.
Ideas from this paper
✗ Failed on benchmark
2026
Compress a transformer KV cache by selecting actual past tokens whose key or hidden-state columns form a stable basis for all cached tokens. Instead of retaining tokens with the largest attention scores or leverage scores independently, compute rank-revealing pivoting of the leading right-singular-vector matrix and retain its pivot columns, then evaluate attention using the representatives plus an optional low-cost residual correction.
Useful7/10
Difficulty5/10
Novelty6/10