{
 "artifacts": null,
 "category": "regularization",
 "description": "Use one fewer point than the exact support threshold for a weighted multi-output linear head. The paper proves that the worst-case multiplicative loss penalty at this budget is only 1+1/(dm^2), giving a principled memory-saving option rather than an arbitrary subset-size heuristic.",
 "formulas_latex": [
  "$$F_{\\mathrm{weighted}}\\bigl(d,m,(m+1)d-1\\bigr)=1+\\frac{1}{dm^{2}}.$$",
  "$$n^{\\star}(d,m)=(m+1)d,$$",
  "$$\\rho(d,m)=1+\\frac{1}{dm^2},\\qquad n_{\\mathrm{exact}}=(m+1)d,\\qquad n_{\\mathrm{near}}=(m+1)d-1.$$",
  "$$A_w=\\sum_{i\\in S}w_i x_i x_i^{\\top},\\qquad B_w=\\sum_{i\\in S}w_i y_i x_i^{\\top},\\qquad \\widehat W_w=B_wA_w^{\\dagger}.$$"
 ],
 "id": 2879,
 "implementation": "Apply this to a frozen feature extractor whose output dimension is d and a multi-class or multi-task linear head with m outputs. At each data-refresh step, compute the full-data reference statistics A=\\sum_i x_ix_i^T and B=\\sum_i y_ix_i^T, then set the support budget to n=(m+1)d-1. Build the support using greedy forward selection: for each candidate, estimate the reduction in the joint residual R=\\|A-A_w\\|_F^2/\\|A\\|_F^2+\\|B-B_w\\|_F^2/\\|B\\|_F^2, add the best candidate, and periodically refit nonnegative weights with NNLS. After reaching n points, perform local swaps and select the support minimizing validation loss of W_w=B_wA_w^{\\dagger}. The paper provides the target factor rho(d,m)=1+1/(dm^2), but the engineer must estimate the realized loss ratio because greedy selection and floating-point arithmetic are outside the exact theorem. Compare against the exact-budget support of (m+1)d points, random weighted sampling, and leverage-score sampling on CIFAR-100 or synthetic rank-r regression. Record full-data excess loss, classification accuracy, support memory, and head-refit time as d and m vary. The predicted signal is negligible degradation for moderate m, with lower memory and solve time than the exact-budget or full-data alternatives.",
 "math_summary": "Theorem 1.2 proves that for all d,m\\ge 1, the weighted selection profile at the near-threshold budget is F_{\\mathrm{weighted}}(d,m,(m+1)d-1)=1+1/(dm^2). Here F_{\\mathrm{weighted}}(d,m,n) is the worst-case ratio between the loss obtained from the best weighted selection of at most n examples and the full-data optimum, over finite datasets with x_i\\in\\mathbb{R}^d and y_i\\in\\mathbb{R}^m. The exact threshold is (m+1)d, so removing one support point has a bounded worst-case degradation. Define rho(d,m)=1+1/(dm^2). This is a theorem for the optimal weighted selection in vector-valued square-loss regression, not an automatic guarantee for a greedy neural-network heuristic; actual excess loss must therefore be measured empirically.",
 "math_tags": [
  "optimization",
  "linear-algebra",
  "convex-analysis",
  "combinatorics"
 ],
 "ml_areas": [
  "embedding",
  "memory",
  "data-augmentation"
 ],
 "paper": {
  "arxiv_id": "2608.30254",
  "arxiv_url": "https://arxiv.org/abs/2608.30254",
  "summary_what_math_gives_to_ml": "The paper gives a sharp structural result for compressing multi-output linear regression datasets: exact recovery of the minimum-Frobenius-norm empirical-risk minimizer is possible with at most (m+1)r weighted examples, where r is the feature rank, and this is necessary in the worst case. This is more specific than generic coreset bounds because the support size scales linearly with both feature rank and output dimension. The most direct transfer is a deterministic weighted replay buffer or linear-probe coreset for neural networks, where a frozen embedding is followed by a repeatedly refit multi-output linear head. The near-threshold profile also supplies a principled accuracy-versus-memory tradeoff when using one fewer point.",
  "title": "Exact Recovery Thresholds for Weighted Data Selection in Vector-Valued Linear Regression",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 4,
  "novelty": 4,
  "usefulness": 6
 },
 "solves": [
  "memory",
  "accuracy",
  "sample-efficiency"
 ],
 "title": "One-Point-Below-Threshold Data Budget",
 "url": "https://synthcore.org/idea/2879/one-point-below-threshold-data-budget",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}