Weak-Type Nonlocal Gradient Regularizer

Integrate this at the loss level for a coordinate MLP \(f_\theta(x)\), such as an implicit image representation, neural field, signed-distance model, or CNN queried at random spatial coordinates. At every training step, sample a minibatch of coordinate pairs \((x_i,y_i)\) from a proposal distribution \(q\), either uniformly over the domain or with a controlled bias toward nearby points. Compute \(d_i=\|x_i-y_i\|_2+\varepsilon\), \(\Delta_i=f_\theta(x_i)-f_\theta(y_i)\), and \(r_i=|\Delta_i|^p/d_i^{p+\gamma}\). Replace the nondifferentiable indicator by \(s_i=\operatorname{sigmoid}(\kappa(r_i-\lambda_t))\), then estimate the regularizer as \(\hat G=\lambda_t B^{-1}\sum_i s_i d_i^{\gamma-N}\) for normalized uniform sampling, or multiply each term by its importance weight if \(q\) is nonuniform. Add \(\alpha\hat G\) to the task loss and backpropagate. Start with moderate \(\kappa\), then anneal it upward so the surrogate approaches the hard threshold. Increase \(\lambda_t\) geometrically from \(\lambda_{\min}\) to \(\lambda_{\max}\). For vector outputs, use the Euclidean norm \(\|f_\theta(x_i)-f_\theta(y_i)\|_2\). The unknown constant \(C_{N,p,\gamma}^{\mathrm{cell}}\) can be omitted and absorbed into \(\alpha\), or estimated by evaluating the functional on linear functions \(u(x)=a^\top x\). The first experiment should use a small 2D coordinate MLP fitting a standard image, comparing this method with no regularization, isotropic total variation, and a conventional fractional pairwise loss at equal pair-evaluation FLOPs. Record validation PSNR, robustness to coordinate noise, gradient variance, and the fraction of active pairs \(s_i>0.5\). Success means better PSNR or noise robustness at equal task loss, with the active-pair fraction decreasing gradually as \(\lambda_t\) increases rather than becoming zero immediately.