LU-Preconditioned Orthogonal Weight Retraction

Integrate this as a periodic retraction for a trainable rectangular matrix W in a linear layer, LoRA adapter, orthogonal bottleneck, or projection head, rather than executing it on every minibatch update. Use W in R^{m x n} with m >= n; for a wide layer apply the procedure to W^T. At each retraction, cast W to fp32 for pivoted LU and HouseholderQR, while testing the CholeskyQR stage in the target precision. Pseudocode: read W; compute [L,U,P] = LU(W) with partial pivoting; compute [H,S] = HouseholderQR(L); set Y = S@U; compute Wpre = triangular_solve(Y, W^T)^T instead of explicitly forming Y^{-1}; calculate G = Wpre^T@Wpre; compute C = chol(G); set Q1 = Wpre@C^{-T}; calculate G2 = Q1^T@Q1 and C2 = chol(G2); set Q = Q1@C2^{-T}; replace W by Q. If Cholesky fails, retry with G + epsilon I and record the failure. Initially stop gradients through the projection and re-estimate all factors after each retraction. Measure ||W^T W-I||_F/sqrt(n), reconstruction error, minimum singular value, Cholesky failures, gradient spikes, training loss, and validation accuracy. Start with a 2-layer MLP or Transformer projection of size 1024 x 1024, initialize singular values across eight orders of magnitude, and train on CIFAR-10 or a small language-model benchmark. Compare unconstrained training, standard QR, polar/Newton-Schulz, and this method every 100-1000 steps in bfloat16. Success is lower orthogonality error and zero numerical breakdowns at equal validation accuracy and similar wall-clock cost.