Order-Derivative Fractional Regularizer
Implementation & benchmark of arXiv:2607.11462 — Regularity for the fractional logarithmic $p$-Laplacian
Source paper: Regularity for the fractional logarithmic $p$-Laplacian arXiv:2607.11462 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Use the derivative of fractional feature energy with respect to its order as a regularizer for intermediate representations. This penalizes unstable scale behavior rather than simply suppressing all high frequencies, so it can preserve useful detail while discouraging uncontrolled changes across spatial scales.
Formulas
Mathematical statement
For $p=2$, define the fractional quadratic energy $E_s(u)=\frac{C_{d,s}}{2}\iint\frac{(u(x)-u(y))^2}{|x-y|^{d+2s}}\,dx\,dy$, where $u$ is a scalar feature map, $d$ is spatial dimension, $s\in(0,1)$ is the fractional order, and $C_{d,s}$ is the normalization constant. Differentiating with respect to $s$ gives $\partial_sE_s(u)=\frac{C_{d,s}}{2}\iint (u(x)-u(y))^2(B_{d,s}-2\log|x-y|)|x-y|^{-d-2s}\,dx\,dy$, with $B_{d,s}=\partial_s\log C_{d,s}$. Because the logarithmic kernel changes sign, $\partial_sE_s$ is signed and is not itself a safe nonnegative penalty. Use the scale-normalized squared quantity $R_{\log}(u)=(\partial_sE_s(u))^2/(E_s(u)+\varepsilon)$, where $\varepsilon>0$ prevents division by zero, and add it to the task loss.
Implementation notes
Apply the regularizer to an intermediate CNN feature map $U\in\mathbb{R}^{B\times C\times H\times W}$, preferably at 1/4 or 1/8 image resolution to limit overhead. Choose a finite offset set $\Delta=\{\delta:0<\|\delta\|_2\le R\}$, use reflect padding, and measure each offset distance in pixels as $r_\delta=\|\delta\|_2$. For each offset compute $D_\delta=U-\mathrm{shift}(U,\delta)$ and $a_\delta=(r_\delta+\epsilon)^{-d-2s}$ with $d=2$. Accumulate $E=\frac12\sum_{\delta\in\Delta}a_\delta\operatorname{mean}(D_\delta^2)$ and $G=\frac12\sum_{\delta\in\Delta}a_\delta(B-2\log(r_\delta+\epsilon))\operatorname{mean}(D_\delta^2)$, then add $\lambda G^2/(E+\epsilon)$ to the task loss. Pseudocode is: initialize $E=G=0$; for each offset, shift $U$, compute $D$, update $E\mathrel{+}=a_\delta\operatorname{mean}(D^2)/2$ and $G\mathrel{+}=a_\delta(B-2\log(r_\delta+\epsilon))\operatorname{mean}(D^2)/2$; return $\mathcal{L}_{task}+\lambda G^2/(E+\epsilon)$. The paper supplies the logarithmic derivative factor; the finite offset quadrature and cutoff radius $R$ are approximations. Initially set $B=0$ to isolate the logarithmic effect; optionally estimate $B$ by finite differences of the chosen discrete normalization, $B\approx[\log C_{d,s+h}-\log C_{d,s-h}]/(2h)$. Test on CIFAR-10 with ResNet-18 against no regularizer and ordinary fractional energy $E_s$, using $s\in\{0.25,0.5,0.75\}$ and $\lambda\in\{10^{-5},10^{-4},10^{-3}\}$. Log clean accuracy, CIFAR-10-C accuracy, feature energy $E$, the signed derivative $G$, gradient norms, and training-loss variance. Success means better corruption robustness or lower feature-scale drift without more than a 1% clean-accuracy loss or materially worse optimization stability.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.