Fractional Nonlocal-Gradient Residual Block

(1) Integration point: insert the branch in a ResNet or ConvNeXt block immediately after the first pointwise projection and before the local spatial convolution. For input x with shape B by C by H by W, compute u = activation(Conv1x1(x)); the nonlocal branch operates on u and returns C output channels. (2) Pseudocode: choose fractional order s, radius R, and offset set Delta = {r in Z^2: 0 < max(|r_1|,|r_2|) <= R}. For each r, shift u by r using reflection padding, compute d_r = shift(u,r) - u, compute q_r = w_r * d_r * r/(||r||_2^(2+s)+epsilon), and sum q_r over offsets. Concatenate the horizontal and vertical components, apply a 1x1 output projection W_o, and return z = x + alpha times the projected tensor. Initialize alpha to zero or 0.01. (3) The paper determines the fractional power-law kernel; the finite radius, padding, quadrature weights, and projections are engineering choices. Start with w_r = 1 and normalize weights so the total kernel norm is one. Optionally parameterize s = sigmoid(s_0), clipped to [0.1,0.95], and learn it. (4) First experiment: compare a baseline ResNet-18 with the modified model on CIFAR-100 and on a small boundary-sensitive segmentation benchmark such as Cityscapes crops. Match parameter count and FLOPs using 1x1 projections. Test R = 2 and R = 4 and s = 0.5, 0.75, and 0.9. Record validation accuracy or mean IoU, boundary F-score, throughput, activation memory, and gradient-norm variance. The hypothesis is improved boundary F-score or accuracy at matched compute, especially for thin structures and textures; failure is no improvement over a local convolution after tuning alpha and normalization.