Fractional Hilbert-Scale Neural Regularizer
Implementation & benchmark of arXiv:2608.15315 — Nonlocal Tikhonov Regularization: Hilbert Scales, Explicit Rates, and the Classical Limit
Source paper: Nonlocal Tikhonov Regularization: Hilbert Scales, Explicit Rates, and the Classical Limit arXiv:2608.15315 ⓘ · analyzed Sep 1, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Add a fractional Sobolev penalty to the spatial output of a neural field or reconstruction CNN, rather than relying only on pixelwise weight decay or total variation. The fractional order s continuously controls high-frequency suppression, allowing an experiment to test whether s less than 1 preserves edges better than the classical integer-order penalty while still reducing noise and unstable oscillations.
Formulas
Mathematical statement
The paper defines the positive self-adjoint operator A_s = I + (-Delta)^s with domain D(A_s^(1/2)) = H_0^s(Omega), where Omega is a bounded Lipschitz domain, -Delta is the restricted Dirichlet Laplacian, and s is in (0,1). Its Tikhonov objective combines the data-fit term ||K u - y_delta||^2 with the quadratic penalty alpha times ||A_s^(1/2)u||^2 = alpha times <u,A_s u>, where K is the forward operator, u is the reconstructed field, y_delta is noisy data, and alpha is positive. The paper uses the isometric change of variables v = A_s^(1/2)u and B = K A_s^(-1/2), reducing the problem to ||Bv-y_delta||^2 + alpha||v||^2. If phi_j are Laplacian eigenfunctions with eigenvalues lambda_j, then A_s phi_j = (1 + lambda_j^s) phi_j, so each spatial mode receives a penalty multiplier 1 + lambda_j^s. For the backward fractional heat example, B*B = A_s^(-1) exp(-2t A_s), where t is the propagation time; in mode j the information factor is (1 + lambda_j^s)^(-1) exp(-2t(1 + lambda_j^s)). The neural adaptation is to apply the same spectral quadratic form to the network output.
Implementation notes
(1) Integration point: apply the regularizer to the final spatial prediction tensor u produced by a coordinate MLP, U-Net, or CNN. Keep the original data loss L_data = ||K(u)-y||_2^2 and add alpha times the fractional spectral energy. For a periodic grid use FFT; for zero-Dirichlet boundaries use a discrete sine transform. (2) Pseudocode: u = net(input); residual = K(u) - y; U = DST(u) or FFT(u); multipliers = 1 + lambda_grid**s; reg = sum(multipliers * abs(U)2) / number_of_grid_points; loss = mean(residual2) + softplus(a) * reg; optimizer.zero_grad(); loss.backward(); optimizer.step(). Here lambda_grid contains the discrete Laplacian eigenvalues and s can be fixed or parameterized as sigmoid(rho) to constrain it to (0,1). (3) Computed from the paper is the spectral multiplier 1 + lambda_k^s and the exact quadratic form; estimated empirically are alpha and s, selected on validation data. For a smoothing K, compute the diagnostic q_k = (1 + lambda_k^s)^(-1) exp(-2t(1 + lambda_k^s)) and report errors only in modes with q_k above a chosen threshold, or use q_k to identify the unrecoverable bandwidth. (4) First experiment: train a small coordinate MLP on 64 by 64 Gaussian deblurring and denoising problems, comparing s values 0.25, 0.5, 0.75, and 1 against weight decay, total variation, and no output regularization. Use identical parameter counts and optimizer settings. Measure test MSE, edge preservation, Fourier error by frequency band, and variance across noise seeds. The expected signal is lower high-frequency error and improved reconstruction stability at equal training FLOPs; for deblurring, success is lower total MSE than s = 1 at the same noise level without oversmoothing sharp edges.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.