RCLUPPr: a new randomized CholeskyQR with LU preconditioning

What the math gives to ML

The paper develops a randomized, LU-preconditioned CholeskyQR pipeline whose transferable asset is stable orthogonalization of severely ill-conditioned tall-skinny matrices under low or mixed precision. The key structural move is to obtain a triangular preconditioner from pivoted LU, orthogonalize the LU factor with Householder QR, and only then apply CholeskyQR2 to a better-conditioned matrix. This suggests a periodic orthogonal retraction for rectangular neural-network weights or adapters, where direct QR or polar iterations in fp16/bfloat16 can lose orthogonality or fail as training makes the matrix ill-conditioned. The likely benefit is a robust mixed-precision orthogonal-weight module or constraint projection rather than a cheaper per-step optimizer.