{
 "artifacts": null,
 "category": "regularization",
 "description": "Add a multiscale Besov penalty to the output of a shallow ReLU^k network, targeting the smoothness threshold that the paper proves is sufficient for finite ridge-variation representation. This suppresses pathological high-frequency output while preserving low-frequency approximation, providing a principled alternative to ordinary parameter weight decay.",
 "formulas_latex": [
  "$$\\mathcal{D}:=\\{\\sigma_k(\\omega\\cdot{}-b):(\\omega,b)\\in\\mathbb{S}^{d-1}\\times[-c,c]\\},\\qquad \\sigma_k(t)=(t)_+^k.$$",
  "$$\\|f\\|_{\\mathcal{L}_1(\\mathcal{D})}:=\\inf\\left\\{\\|\\mu\\|_{\\mathrm{TV}}:f(x)=\\int_{\\mathbb{S}^{d-1}\\times[-c,c]}\\sigma_k(\\omega\\cdot x-b)\\,d\\mu(\\omega,b)\\right\\}.$$",
  "$$B^s_{p,q}(\\Omega)\\hookrightarrow\\mathcal{L}_1(\\mathcal{D})\\quad\\text{if }s\\ge k+d/p\\ (0\u003cq\\le1),\\qquad s\u003ek+d/p\\ (1\u003cq\\le\\infty).$$",
  "$$R_{s,p,q}(f):=\\left(\\sum_{j=0}^{J}2^{jsq}\\|\\Delta_jf\\|_{L^p(\\Omega)}^q\\right)^{1/q},\\qquad \\mathcal{L}=\\mathcal{L}_{\\mathrm{task}}+\\lambda R_{s,p,q}(f_\\theta).$$"
 ],
 "id": 2986,
 "implementation": "Integrate the penalty at the output of a shallow ReLU^k MLP, before reduction of the task loss. Assume inputs lie in a bounded box in \\(\\mathbb{R}^d\\). At every training step, evaluate the network on a regular grid or fixed random lattice of domain points and reshape scalar outputs into a grid. Apply an FFT, multiply by precomputed annular masks \\(M_j(\\xi)\\) selecting frequencies near \\(2^j\\), and inverse FFT to obtain \\(\\Delta_jf\\). For each band compute \\(r_j=2^{js}(\\operatorname{mean}_x|\\Delta_jf(x)|^p+\\epsilon)^{1/p}\\), then set \\(R=(\\sum_{j=0}^{J}r_j^q)^{1/q}\\) or use \\(R=\\max_jr_j\\) for \\(q=\\infty\\). Optimize \\(\\mathcal{L}_{task}+\\lambda R\\). Start with \\(p=1\\), \\(q=1\\), and \\(s\\) just above \\(k+d/p\\); compare with a lower-smoothness penalty and weight decay. The paper supplies the embedding threshold; grid discretization, padding, frequency masks, \\(\\lambda\\), and the cutoff \\(J\\) are empirical choices. First test a width-256 shallow network on 2D regression targets made from smooth functions plus localized bumps. Compare no regularization, weight decay, and the Besov penalty at matched training FLOPs. Measure test MSE, high-frequency energy, sensitivity to input noise, and error at fixed width. Success is lower test error or perturbation sensitivity at equal training loss, with a reduced high-frequency tail rather than merely slower fitting.",
 "math_summary": "The paper defines \\(\\sigma_k(t)=(t)_+^k\\) and the normalized ridge dictionary \\(\\mathcal{D}=\\{\\sigma_k(\\omega\\cdot{}-b):(\\omega,b)\\in\\mathbb{S}^{d-1}\\times[-c,c]\\}\\), where \\(\\Omega\\subset\\mathbb{R}^d\\) is bounded and Lipschitz, \\(\\omega\\) is a unit direction, and \\(b\\) is a bounded offset. Its variation norm is \\(\\|f\\|_{\\mathcal{L}_1(\\mathcal{D})}=\\inf\\{\\|\\mu\\|_{\\mathrm{TV}}:f(x)=\\int\\sigma_k(\\omega\\cdot x-b)\\,d\\mu(\\omega,b)\\}\\), with \\(\\mu\\) a signed measure and \\(\\|\\mu\\|_{\\mathrm{TV}}\\) its total variation. The forward embedding states \\(B^s_{p,q}(\\Omega)\\hookrightarrow\\mathcal{L}_1(\\mathcal{D})\\) for \\(s\\ge k+d/p\\) when \\(0\u003cq\\le1\\), and for \\(s\u003ek+d/p\\) when \\(1\u003cq\\le\\infty\\). We approximate the Besov norm using a dyadic Littlewood--Paley decomposition \\(f=\\sum_j\\Delta_jf\\), where \\(\\Delta_j\\) contains frequencies of order \\(2^j\\), and penalize \\(R_{s,p,q}(f)=\\left(\\sum_j2^{jsq}\\|\\Delta_jf\\|_{L^p}^q\\right)^{1/q}\\).",
 "math_tags": [
  "functional-analysis",
  "harmonic-analysis",
  "approximation-theory"
 ],
 "ml_areas": [
  "mlp",
  "regularization",
  "training-dynamics"
 ],
 "paper": {
  "arxiv_id": "2609.00680",
  "arxiv_url": "https://arxiv.org/abs/2609.00680",
  "summary_what_math_gives_to_ml": "The paper establishes a sharp correspondence between shallow ReLU^k ridge networks with bounded coefficient variation and isotropic Besov smoothness. Its transferable asset is a mathematically grounded complexity measure: total variation of the representing measure controls both approximation structure and frequency regularity, rather than merely penalizing Euclidean parameter magnitudes. This supports two practical mechanisms: multiscale Besov regularization of network outputs and active-set neuron growth or pruning under an explicit variation budget. The strongest initial validation should use low-dimensional function regression, where Fourier spectra, active width, and approximation error can be measured directly.",
  "title": "Sharp embeddings between quasi-Banach Besov spaces and shallow ReLU variation spaces",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 4,
  "novelty": 6,
  "usefulness": 6
 },
 "solves": [
  "stability",
  "generalization",
  "accuracy"
 ],
 "title": "Besov spectral regularization for shallow ReLU",
 "url": "https://synthcore.org/idea/2986/besov-spectral-regularization-for-shallow-relu",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}