Dual-consistent feature–sample pruning

Integrate the method into a convex subproblem: freeze an encoder, form embeddings h_i, and train a sparse linear probe or adapter w with L(w)=Σ_i f_i((Aw)_i)+λ||w||_1, where row i of A is the embedding of sample i and column j is an embedding coordinate or adapter feature. At each pruning round: (1) warm-start the convex head and run proximal-gradient updates; (2) obtain a dual estimate y from the residual or loss-gradient map and project it onto the known dual-feasible set; (3) calculate feature scores c_j=|A_{:j}^T y|; if a primal–dual gap gives radius R, permanently remove feature j only when c_j+R||A_{:j}||_2<λ; otherwise rank features by c_j and remove the lowest-scoring fraction as an explicitly heuristic variant; (4) calculate sample scores r_i=f_i*(y_i)−min_t f_i*(t), normalize them by loss scale, and remove the lowest-scoring examples subject to a validation safeguard; (5) apply both masks to A and continue warm-started optimization. The mathematical components are the FR dual, the KKT feature rule, the safe-radius correction, and the row/column equivariance. Estimate R from the current primal–dual gap and the local strong-convexity or Lipschitz constants; estimate sample thresholds empirically. First test CIFAR-10 with a frozen ResNet-18 and sparse logistic-regression head. Compare magnitude feature pruning plus random sample pruning against dual-consistent pruning at 25%, 50%, and 75% reduction. Record head FLOPs, retained feature dimension, retained samples, primal–dual gap, convergence speed, and validation accuracy. A positive result is equal accuracy with fewer head FLOPs and examples, or faster loss descent at equal accuracy; then transfer selected embedding/channel masks to full-network fine-tuning.