{
 "artifacts": null,
 "category": "memory",
 "description": "Compress a dataset for a frozen neural embedding and multi-output linear head into at most (m+1)r weighted examples while preserving the full-data minimum-Frobenius-norm least-squares solution. This can make repeated linear probing, classifier refitting, or replay-buffer training much smaller without changing the optimum in exact arithmetic.",
 "download_zip": "https://synthcore.org/download/1124",
 "formulas_latex": [
  "$$n^{\\star}(d,m)=\\min\\{n:F_{\\mathrm{weighted}}(d,m,n)=1\\}=(m+1)d.$$",
  "$$|S|\\le (m+1)r\\quad\\Longrightarrow\\quad \\text{some weighted selection recovers a full-data optimum exactly},$$",
  "$$A_w=\\sum_{i\\in S}w_i x_i x_i^{\\top},\\qquad B_w=\\sum_{i\\in S}w_i y_i x_i^{\\top},\\qquad W_w=B_wA_w^{\\dagger}.$$",
  "$$\\min_W\\sum_i\\|Wx_i-y_i\\|_2^2\\quad\\leadsto\\quad \\min_W\\sum_{i\\in S}w_i\\|Wx_i-y_i\\|_2^2.$$"
 ],
 "id": 2878,
 "implementation": "Integrate this after a frozen backbone and before a linear classifier or regression head. Given embeddings x_i\\in\\mathbb{R}^d and targets y_i\\in\\mathbb{R}^m, compute a numerical rank r using an SVD, retaining singular values above 10^{-6} times the largest singular value. Compute full statistics A=\\sum_i x_ix_i^T and B=\\sum_i y_ix_i^T. Initialize a support S with r linearly independent examples and positive weights. Form each atom z_i=[\\operatorname{vech}(x_ix_i^T);\\operatorname{vec}(y_ix_i^T)]. While the support is larger than the target, find a null vector c of the active atom matrix Z_S, choose a sign and step t so that w\\leftarrow w-tc remains nonnegative, and delete any zero-weight atom. Because Z_Sc=0, this preserves the aggregate statistics. If generic elimination stops above (m+1)r, use support exchange: sample candidates outside S, solve nonnegative least squares to match [A;B], and accept swaps reducing support while keeping relative residual below 10^{-6}. The theorem supplies the support target; rank thresholds, NNLS tolerances, and exchange heuristics are empirical. Test on CIFAR-10 or a small multi-target regression task with a frozen ResNet-18 or MLP embedding, comparing the compressed head with the full-data head and a random subset of equal size. Measure support size, loss error, probe-refit time, and peak memory. Success means support near (m+1)r, relative loss error below 10^{-5}, and lower refit cost.",
 "math_summary": "The paper defines the exact-recovery threshold by n^{\\star}(d,m)=\\min\\{n:F_{\\mathrm{weighted}}(d,m,n)=1\\} and proves that n^{\\star}(d,m)=(m+1)d in the worst case. More generally, for a dataset whose feature matrix has rank r, some (m+1)r weighted points recover a full-data optimum exactly. For feature vectors x_i\\in\\mathbb{R}^d, targets y_i\\in\\mathbb{R}^m, and nonnegative weights w_i, define A_w=\\sum_i w_i x_i x_i^{\\top} and B_w=\\sum_i w_i y_i x_i^{\\top}. The minimum-Frobenius-norm weighted least-squares solution is W_w=B_wA_w^{\\dagger}, where A_w^{\\dagger} is the Moore-Penrose pseudoinverse, provided the weighted normal equations represent the same optimum. The implementation searches for a sparse nonnegative representation preserving the sufficient statistics needed by the quadratic objective.",
 "math_tags": [
  "linear-algebra",
  "convex-analysis",
  "combinatorics"
 ],
 "ml_areas": [
  "embedding",
  "memory",
  "fine-tuning"
 ],
 "paper": {
  "arxiv_id": "2608.30254",
  "arxiv_url": "https://arxiv.org/abs/2608.30254",
  "summary_what_math_gives_to_ml": "The paper gives a sharp structural result for compressing multi-output linear regression datasets: exact recovery of the minimum-Frobenius-norm empirical-risk minimizer is possible with at most (m+1)r weighted examples, where r is the feature rank, and this is necessary in the worst case. This is more specific than generic coreset bounds because the support size scales linearly with both feature rank and output dimension. The most direct transfer is a deterministic weighted replay buffer or linear-probe coreset for neural networks, where a frozen embedding is followed by a repeatedly refit multi-output linear head. The near-threshold profile also supplies a principled accuracy-versus-memory tradeoff when using one fewer point.",
  "title": "Exact Recovery Thresholds for Weighted Data Selection in Vector-Valued Linear Regression",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 6,
  "novelty": 5,
  "usefulness": 7
 },
 "solves": [
  "memory",
  "speedup",
  "sample-efficiency"
 ],
 "title": "Exact Multi-Output Linear-Probe Coreset",
 "url": "https://synthcore.org/idea/2878/exact-multi-output-linear-probe-coreset",
 "verification": {
  "peer_reviewed": false,
  "status": "queued_mechanism",
  "status_label": "Queued — mechanism check",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}