Confidence-Tested LoRA Pruning

Integrate the method into LoRA fine-tuning immediately before the optimizer update and at periodic rank-reduction checkpoints. For each adapted weight matrix W_ell, write the LoRA update as A_ell B_ell = sum_j a_{ell,j} b_{ell,j}^T, where a_{ell,j} is column j of A_ell, b_{ell,j}^T is row j of B_ell, and u_{ell,j} = a_{ell,j} b_{ell,j}^T is the rank-one direction. At every training step or every few steps, compute the detached scalar x_{t,ell,j} = inner_product(grad(W_ell), u_{ell,j}); alternatively multiply by the current learning rate to measure predicted first-order loss decrease. Store a short history for every component. At a checkpoint, compute the sample mean s_hat, autocovariances gamma_hat_k, long-run variance v_hat using the displayed Newey-West estimator with lag cutoff K, then calculate SE, z, and p. Choose Delta as a fixed fraction of the median positive component score or tune it on a validation set. Rank components by increasing p-value or by increasing lower confidence bound, and delete the weakest components until the desired total rank is reached by removing the corresponding columns of A and rows of B. Use a warmup period before testing and impose a minimum rank per layer. The paper supplies the hypothesis-testing interpretation and optimizer-trajectory asymptotics; the contribution proxy and finite-sample variance estimator are practical approximations. First test on DeBERTa-v3-base with rank-16 LoRA on MNLI, comparing uniform LoRA and AdaLoRA at matched trainable parameter counts. Record validation accuracy, loss after pruning, rank allocation, and allocation variance over at least three seeds. Success means equal or better accuracy at the same final rank, lower sensitivity to seed, or a smaller rank at matched accuracy.