{
 "artifacts": null,
 "category": "architecture",
 "description": "Treat a finite shallow network as a discrete signed measure over ridge atoms and grow or prune neurons according to their contribution to total variation. This turns width selection into an atomic approximation procedure: add neurons correlated with the current residual and remove coefficients that consume budget without contributing materially.",
 "formulas_latex": [
  "$$f_\\mu(x)=\\int_{\\mathbb{S}^{d-1}\\times[-c,c]}\\sigma_k(\\omega\\cdot x-b)\\,d\\mu(\\omega,b).$$",
  "$$\\mu=\\sum_{i=1}^{m}a_i\\delta_{(\\omega_i,b_i)},\\qquad f_\\mu(x)=\\sum_{i=1}^{m}a_i(\\omega_i\\cdot x-b_i)_+^k,\\qquad \\|\\mu\\|_{\\mathrm{TV}}=\\sum_{i=1}^{m}|a_i|.$$",
  "$$\\|f\\|_{\\mathcal{L}_1(\\mathcal{D})}:=\\inf_{\\mu:f=f_\\mu}\\|\\mu\\|_{\\mathrm{TV}},\\qquad \\mathcal{L}_1(\\mathcal{D})\\hookrightarrow B^{k+1}_{p,2}(\\Omega)\\quad(0\u003cp\u003c1).$$",
  "$$\\max_{(\\omega,b)\\in\\mathbb{S}^{d-1}\\times[-c,c]}\\left|\\frac{1}{n}\\sum_{r=1}^{n}e_r\\,\\sigma_k(\\omega\\cdot x_r-b)\\right|.$$"
 ],
 "id": 2987,
 "implementation": "Replace a dense shallow layer by an active set of neurons \\((\\omega_i,b_i,a_i)\\). Maintain minibatch residuals \\(e_r=f_\\theta(x_r)-y_r\\). Every \\(T\\) optimizer steps, sample \\(K\\) random unit directions and offsets, evaluate the absolute residual correlation \\(c(\\omega,b)=|n^{-1}\\sum_re_r\\sigma_k(\\omega\\cdot x_r-b)|\\), and add the best atom. Optionally refine its \\((\\omega,b)\\) by a few gradient-ascent steps on \\(c\\). Initialize its coefficient as \\(a=\\eta_a\\operatorname{sign}(n^{-1}\\sum_re_r\\sigma_k(\\omega\\cdot x_r-b))\\), then jointly optimize all active parameters. Every \\(T_{prune}\\) steps, delete atoms with \\(|a_i|\u003c\\tau\\), and enforce a total variation budget \\(V=\\sum_i|a_i|\\le V_{max}\\) by soft-thresholding or projection onto the \\(\\ell^1\\) ball. The paper supplies the dictionary, TV complexity, and reverse Besov regularity claim; sampling size, coefficient step, pruning threshold, and budget are empirical. First compare active-set growth against a fixed width-256 shallow MLP and magnitude pruning on 2D regression with smooth and localized targets. Log validation error versus active neuron count, \\(V\\), wall-clock time, and inference memory. Success is reaching a target error with fewer active neurons and lower memory, while the smooth target should exhibit smaller final variation and the localized target should require additional atoms rather than being incorrectly over-pruned.",
 "math_summary": "For \\(\\omega_i\\in\\mathbb{S}^{d-1}\\), offsets \\(b_i\\in[-c,c]\\), and coefficients \\(a_i\\in\\mathbb{R}\\), a finite network corresponds to \\(\\mu=\\sum_{i=1}^ma_i\\delta_{(\\omega_i,b_i)}\\), giving \\(f_\\mu(x)=\\sum_{i=1}^ma_i(\\omega_i\\cdot x-b_i)_+^k\\). The measure's total variation is \\(\\|\\mu\\|_{\\mathrm{TV}}=\\sum_i|a_i|\\), while the paper's atomic variation norm is the infimum of this quantity over all measures representing the same function: \\(\\|f\\|_{\\mathcal{L}_1(\\mathcal{D})}=\\inf_{\\mu:f=f_\\mu}\\|\\mu\\|_{\\mathrm{TV}}\\). The reverse embedding for \\(0\u003cp\u003c1\\) is \\(\\mathcal{L}_1(\\mathcal{D})\\hookrightarrow B^{k+1}_{p,2}(\\Omega)\\), so bounded variation imposes Besov smoothness \\(k+1\\) and fine index 2. We use \\(V=\\sum_i|a_i|\\) as an explicit complexity and memory-control variable. New atoms are selected by correlation with residuals over the same sphere-and-offset dictionary.",
 "math_tags": [
  "measure-theory",
  "functional-analysis",
  "approximation-theory",
  "harmonic-analysis"
 ],
 "ml_areas": [
  "mlp",
  "pruning",
  "memory",
  "initialization"
 ],
 "paper": {
  "arxiv_id": "2609.00680",
  "arxiv_url": "https://arxiv.org/abs/2609.00680",
  "summary_what_math_gives_to_ml": "The paper establishes a sharp correspondence between shallow ReLU^k ridge networks with bounded coefficient variation and isotropic Besov smoothness. Its transferable asset is a mathematically grounded complexity measure: total variation of the representing measure controls both approximation structure and frequency regularity, rather than merely penalizing Euclidean parameter magnitudes. This supports two practical mechanisms: multiscale Besov regularization of network outputs and active-set neuron growth or pruning under an explicit variation budget. The strongest initial validation should use low-dimensional function regression, where Fourier spectra, active width, and approximation error can be measured directly.",
  "title": "Sharp embeddings between quasi-Banach Besov spaces and shallow ReLU variation spaces",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 5,
  "usefulness": 5
 },
 "solves": [
  "memory",
  "scalability",
  "generalization"
 ],
 "title": "Variation-budget neuron growth",
 "url": "https://synthcore.org/idea/2987/variation-budget-neuron-growth",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}