Sharp fractional interpolation envelope for feature maps
Implementation & benchmark of arXiv:2608.08686 — Sharp homogeneous Gagliardo--Nirenberg inequalities with applications to normalized solutions for a generalized MMT-type equation
Source paper: Sharp homogeneous Gagliardo--Nirenberg inequalities with applications to normalized solutions for a generalized MMT-type equation arXiv:2608.08686 ⓘ · analyzed Sep 1, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Add a scale-invariant Gagliardo–Nirenberg ratio penalty to intermediate CNN or spatial neural-network feature maps. The penalty discourages representations with unusually large low-order fractional gradients relative to their amplitude and high-order energy, providing a single mathematically coupled constraint instead of separately weighted total-variation and Sobolev penalties. Apply it only to selected layers and estimate the reference sharp constant from clean baseline activations.
Formulas
Mathematical statement
The paper defines the homogeneous fractional derivative $D^s=(-\Delta)^{s/2}$ as the Fourier multiplier with symbol $|\xi|^s$. Under $1<p,q<\infty$, $0<s_1\leq s_2$, $s_1/s_2<\theta<1$, and the scaling relation $\frac{1}{q}-\frac{s_1}{d}=(1-\theta)\frac{1}{p}+\theta\left(\frac{1}{2}-\frac{s_2}{d}\right)$, the sharp constant is $C_{\mathrm{BFV}}:=\sup_{\phi\neq 0}\frac{\|D^{s_1}\phi\|_{L^q}}{\|\phi\|_{L^p}^{1-\theta}\|D^{s_2}\phi\|_{L^2}^{\theta}}$, with $\phi\in\dot H^{s_2}(\mathbb R^d)\cap L^p(\mathbb R^d)$. Therefore admissible fields satisfy $\|D^{s_1}\phi\|_q\leq C_{\mathrm{BFV}}\|\phi\|_p^{1-\theta}\|D^{s_2}\phi\|_2^\theta$. Here $d$ is the spatial dimension, $p$ and $q$ are integrability exponents, $s_1$ and $s_2$ are fractional derivative orders, and $\theta$ is fixed by scaling. For a discrete feature map $h$, use the periodic FFT approximation $\widehat{D^s h}(\xi)=|\xi|^s\hat h(\xi)$ and define $r(h)=\frac{\|D^{s_1}h\|_q}{(\|h\|_p+\varepsilon)^{1-\theta}(\|D^{s_2}h\|_2+\varepsilon)^\theta}$. The adapted regularizer is $\mathcal R(h)=[\max(0,\log r(h)-\log\widehat C)]^2$, where $\widehat C$ is an empirical upper-bound estimate rather than a claimed exact sharp constant.
Implementation notes
Integrate the method after an intermediate spatial feature tensor in a CNN, U-Net, or Fourier-style spatial encoder. Let $h\in\mathbb R^{B\times C\times H\times W}$, and compute the penalty on a random subset of channels and examples to reduce overhead. Use $d=2$, $p=q=2$, $s_1=1/2$, and $s_2=1$; the scaling equation then gives $\theta=1/2$. For each selected feature map, subtract its spatial mean, compute a 2-D orthonormal FFT, multiply Fourier coefficients by $(k_x^2+k_y^2)^{s/2}$ for $s=s_1$ and $s=s_2$, inverse-transform, and calculate the required norms. The training step is: h=feature(x); F=fft2(h-mean(h)); hs1=ifft2(freq**s1*F); hs2=ifft2(freq**s2*F); ratio=norm(hs1,2)/((norm(h,2)+eps)**(1-theta)*(norm(hs2,2)+eps)theta); reg=relu(log(ratio)-log(C_hat))2; loss=task_loss+lambda*reg; loss.backward(); optimizer.step(). Estimate $\widehat C$ during an unregularized warm-up by collecting ratios and taking their 99th percentile, then optionally update it with a slow exponential moving maximum. The Gagliardo–Nirenberg inequality and scaling relation are taken from the paper; FFT boundary conditions, hinge activation, percentile estimation, and subsampling are engineering choices. First test on CIFAR-10 using ResNet-18, comparing no regularizer, total variation, and a standard Laplacian penalty at matched FLOPs. Sweep $\lambda\in\{10^{-4},10^{-3},10^{-2}\}$ and measure clean accuracy, CIFAR-10-C accuracy, activation spectral tails, gradient-norm variance, and loss-spike frequency. Success requires improved corruption accuracy or fewer instability events without more than a 0.2 percentage-point clean-accuracy loss.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.