{
 "artifacts": null,
 "category": "regularization",
 "description": "Add a regularizer that rewards each neuron's expected absolute response to random sign perturbations, normalized by the neuron's l2 norm so ordinary weight scaling cannot trivially increase the objective. Use the paper's distance-sensitive Khintchine lower bound to penalize filters close to the two-coordinate extremal set, promoting distributed and perturbation-stable feature extraction.",
 "formulas_latex": [
  "$$K_n(\\omega)=\\mathbb{E}_{\\epsilon}\\left|\\sum_{i=1}^{n}\\omega_i\\epsilon_i\\right|,$$",
  "$$K_n(\\omega)\\geq\\frac{1}{\\sqrt{2}}\\|\\omega\\|_2,$$",
  "$$K_n(\\omega)\\geq\\frac{1}{\\sqrt{2}}+g(\\tau)\\tau,\\qquad g(t)=\\frac{\\sqrt{1-t^{2}/4}-t}{2\\sqrt{2}},\\qquad \\|\\omega\\|_2=1,$$",
  "$$\\tau(u)=\\sqrt{2-\\sqrt{2}\\left(|u|_{(1)}+|u|_{(2)}\\right)},\\qquad B(u)=\\frac{1}{\\sqrt{2}}+\\frac{\\tau(u)}{36},\\qquad u=\\frac{\\omega}{\\|\\omega\\|_2+\\varepsilon}.$$"
 ],
 "id": 2778,
 "implementation": "Integrate this at the weight tensor of an MLP layer or CNN convolution, independently for each output channel. For every filter \\(w_j\\), flatten all input-coordinate dimensions into one vector, compute \\(r_j=\\|w_j\\|_2+10^{-8}\\) and \\(u_j=w_j/r_j\\), then find the two largest entries of \\(|u_j|\\). Set \\(\\tau_j=\\sqrt{\\max(0,2-\\sqrt{2}(a_j+b_j))}\\), where \\(a_j,b_j\\) are the top-two magnitudes, and compute \\(B_j=1/\\sqrt{2}+\\tau_j/36\\). During training, estimate the actual Khintchine quantity with \\(S\\) random sign vectors: \\(\\widehat K_j=S^{-1}\\sum_{s=1}^{S}|u_j^T\\epsilon^{(s)}|\\), using \\(S=4\\) and resampling signs each batch. Add \\(L_{anti}=\\lambda\\,\\mathrm{mean}_j[\\max(0,B_{target}-\\widehat K_j)]^2\\), with \\(B_{target}\\) chosen between \\(1/\\sqrt{2}\\) and the empirical 75th percentile of initial \\(\\widehat K_j\\). A cheaper deterministic variant uses \\(L_{cert}=\\lambda\\,\\mathrm{mean}_j[\\max(0,B_{target}-B_j)]^2\\), requiring only top-two selection. Do not normalize the forward weights and do not apply the penalty to biases. First test on CIFAR-10 with a ResNet-18 and on a two-layer MLP, comparing ordinary weight decay against the proposed penalty at matched parameter count and training schedule, with \\(\\lambda\\in\\{10^{-4},10^{-3},10^{-2}\\}\\). Measure clean accuracy, accuracy after independently flipping 10% of input signs, calibration, mean \\(\\widehat K_j\\), and the fraction of filters with \\(\\tau_j\u003c0.1\\). Success requires improved corrupted-input accuracy or calibration at matched clean accuracy, fewer near-extremal filters, and less than 5% training overhead.",
 "math_summary": "For independent Rademacher variables \\(\\epsilon_i\\in\\{-1,+1\\}\\), define \\(K_n(\\omega)=\\mathbb{E}_{\\epsilon}|\\sum_{i=1}^n\\omega_i\\epsilon_i|\\), where \\(\\omega\\in\\mathbb{R}^n\\) is a neuron or convolutional filter. The classical bound is \\(K_n(\\omega)\\geq\\|\\omega\\|_2/\\sqrt{2}\\), with equality after \\(\\|\\omega\\|_2=1\\) only for vectors having exactly two nonzero coordinates of equal magnitude, up to signs and permutation. The extracted theorem gives, for normalized \\(\\|\\omega\\|_2=1\\) and distance \\(\\tau\\) from this extremal set, \\(K_n(\\omega)\\geq1/\\sqrt{2}+g(\\tau)\\tau\\), where \\(g(t)=(\\sqrt{1-t^2/4}-t)/(2\\sqrt{2})\\). In the first regime \\(0\\leq\\tau\\leq d_4\\), the paper proves \\(g(\\tau)\\geq g(d_4)\u003e0.056\u003e1/36\\). For implementation, set \\(u=\\omega/(\\|\\omega\\|_2+\\varepsilon)\\), let \\(|u|_{(1)}\\) and \\(|u|_{(2)}\\) denote the two largest absolute coordinates, and define the distance to the signed/permuted extremal set as \\(\\tau(u)=\\sqrt{2-\\sqrt{2}(|u|_{(1)}+|u|_{(2)})}\\). This follows because the maximum inner product with a normalized two-sparse equal-magnitude vector is \\((|u|_{(1)}+|u|_{(2)})/\\sqrt{2}\\). A conservative theorem-derived certificate is \\(B(u)=1/\\sqrt{2}+\\tau(u)/36\\).",
 "math_tags": [
  "probability",
  "geometry",
  "inequalities"
 ],
 "ml_areas": [
  "mlp",
  "cnn",
  "regularization"
 ],
 "paper": {
  "arxiv_id": "2608.27908",
  "arxiv_url": "https://arxiv.org/abs/2608.27908",
  "summary_what_math_gives_to_ml": "The paper provides a quantitative certificate that a Rademacher linear form cannot remain close to its minimum expected absolute magnitude unless its weight vector is close to a two-coordinate, equal-magnitude extremizer. This can be transferred into a neuron regularizer that discourages degenerate filters or feature projections whose energy is concentrated in only two coordinates, while remaining invariant to signs and permutations. The practical implementation is to estimate expected absolute responses under random sign perturbations and combine them with the paper's explicit distance-sensitive lower bound, yielding a falsifiable feature-diversity and robustness experiment.",
  "title": "A Two-regime Khintchine Inequality and an Improved Bound on the Degree-1 Fourier Weight for Linear Threshold Functions",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 3,
  "novelty": 8,
  "usefulness": 5
 },
 "solves": [
  "accuracy",
  "stability",
  "generalization"
 ],
 "title": "Khintchine anti-degeneracy regularizer",
 "url": "https://synthcore.org/idea/2778/khintchine-anti-degeneracy-regularizer",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}