Unverified 2026

Weighted CKN feature-stability regularizer

Implementation & benchmark of arXiv:2608.16803 — Sharp $L^2$-Caffarelli--Kohn--Nirenberg and weighted Poincaré inequalities on half-spaces and orthants and their stability

Usefulness5/10

Difficulty5/10

Novelty5/10

Regularization Stability Generalization Accuracy Regularization Training Mlp Transformer Functional analysis Inequalities Harmonic analysis Geometry

Source paper: Sharp $L^2$-Caffarelli--Kohn--Nirenberg and weighted Poincaré inequalities on half-spaces and orthants and their stability arXiv:2608.16803 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Apply a weighted Caffarelli–Kohn–Nirenberg deficit to selected intermediate feature channels. The penalty discourages features that obtain large weighted responses only by becoming sharply localized or highly sensitive to small input perturbations. It can be evaluated with input-Jacobian estimates and added to the ordinary task loss.

Formulas

$$\left\|\frac{u}{|x|^{a}}\right\|_{L^{2}}\left\|\frac{\nabla u}{|x|^{b}}\right\|_{L^{2}}\geq C(n,a,b)\left\|\frac{u}{|x|^{(a+b+1)/2}}\right\|_{L^{2}}^{2}$$

$$A=\mathbb E[|x|^{-2a}u(x)^2],\qquad B=\mathbb E[|x|^{-2b}\|\nabla_xu(x)\|_2^2],\qquad D=\mathbb E[|x|^{-(a+b+1)}u(x)^2],\qquad \sqrt{AB}\ge C(n,a,b)D$$

$$d\mu(x)=e^{-\delta|x|^{\tau}}|x|^{\beta}\left(\prod_{i=n-k+1}^{n}x_i^{2}\right)dx$$

$$\ell_{\mathrm{CKN}}=\operatorname{softplus}\left(\log(C_{ref}D+\varepsilon)-\frac12\log(AB+\varepsilon)\right)$$

Mathematical statement

The paper's central weighted inequality is $\|u/|x|^{a}\|_{L^{2}}\|\nabla u/|x|^{b}\|_{L^{2}}\ge C(n,a,b)\|u/|x|^{(a+b+1)/2}\|_{L^{2}}^{2}$. Here $u$ is a scalar function, $x\in\mathbb R^n$, $a,b$ are weight exponents, and $C(n,a,b)$ is the sharp domain- and dimension-dependent constant. Define $A=\mathbb E[|x|^{-2a}u(x)^2]$, $B=\mathbb E[|x|^{-2b}\|\nabla_xu(x)\|_2^2]$, and $D=\mathbb E[|x|^{-(a+b+1)}u(x)^2]$; then the inequality is $\sqrt{AB}\ge C(n,a,b)D$. The proposed loss is a soft violation penalty, $\ell_{CKN}=\operatorname{softplus}(\log(C_{ref}D+\varepsilon)-\tfrac12\log(AB+\varepsilon))$, where $C_{ref}$ is the theorem's sharp constant when the input domain matches the paper, or a conservative calibrated reference constant otherwise. The paper also considers weighted measures $d\mu(x)=e^{-\delta|x|^{\tau}}|x|^{\beta}(\prod_{i=n-k+1}^{n}x_i^2)dx$, with $\delta,\tau,\beta$ and $k$ defining radial and orthant weighting; this can be used as an alternative sampling measure.

Implementation notes

(1) Integration point: select an intermediate hidden representation in a small MLP or transformer, after a residual block and before the next normalization layer. For each example $x_j\in\mathbb R^n$, choose a scalar feature $u_j=v^\top h(x_j)$, where $v$ is a fixed random projection or one selected channel. Use 4–16 projections per minibatch. Normalize inputs first and define $r_j=(\|x_j\|_2^2+\varepsilon)^{1/2}$. (2) Pseudocode: compute $u_j=v^\top h(x_j)$; obtain $g_j=\nabla_{x_j}u_j$ by reverse-mode autodiff; set $A=\operatorname{mean}(r_j^{-2a}u_j^2)$, $B=\operatorname{mean}(r_j^{-2b}\|g_j\|_2^2)$, and $D=\operatorname{mean}(r_j^{-(a+b+1)}u_j^2)$; compute $L_{reg}=\operatorname{softplus}(\log(C_{ref}D+\varepsilon)-0.5\log(AB+\varepsilon))$; optimize $L=L_{task}+\lambda L_{reg}$. Use stop-gradient on weights and clip $r_j$ to robust batch quantiles. (3) The weighted norm definitions and deficit come from the CKN inequality. The Jacobian is estimated exactly for each scalar projection or with Hutchinson probes if using vector-valued features. Estimate $C_{ref}$ from the sharp theorem where applicable; otherwise choose the 5th percentile of $\sqrt{AB}/(D+\varepsilon)$ over a stable pretrained baseline, then keep it fixed. (4) First experiment: train a 2-layer CIFAR-10 MLP and a 6-layer WikiText-2 transformer, comparing this penalty with weight decay and ordinary Jacobian regularization at matched FLOPs. Measure clean accuracy or perplexity, input-gradient norms, loss sharpness, and Gaussian-noise/corruption accuracy. The falsifiable success criterion is improved corruption robustness or reduced worst-decile Jacobian norm at matched clean performance; failure is any gain that disappears after matching gradient-computation overhead.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.