Convergence rates for pivoted QR and LU

arXiv:2607.26863 2026 Memory 1 ideas extracted · analyzed Aug 31, 2026

What the math gives to ML

This paper gives a constructive way to build low-rank approximations by greedily selecting columns and repeatedly removing their contribution. Its transferable asset is not QR itself, which is standard numerical linear algebra, but the determinant-based explanation of why greedy pivots remain effective when singular values decay algebraically or geometrically. A promising neural-network use is adaptive landmark selection for compressing attention or kernel matrices: choose tokens by residual energy rather than uniformly or randomly, then represent all attention columns using the selected landmarks. The method provides an explicit residual-based stopping rule and a falsifiable accuracy-versus-rank tradeoff.

Ideas from this paper

Unverified 2026

Residual-pivoted landmark attention

Replace uniform or random landmark selection in low-rank attention with approximate greedy pivoted QR selection on the attention matrix. At each step, select the key whose residual attention column has the largest norm, orthogonalize it against previously selected columns, and stop when the residual energy falls below a target tolerance. Use the selected columns as an adaptive attention basis and reconstruct the full attention output through a small coefficient matrix.

Useful6/10
Difficulty6/10
Novelty5/10
Paper: Convergence rates for pivoted QR and LU arXiv:2607.26863