Fractional Sign-Oscillation Penalty

Integrate the regularizer after a graph-neural-network layer, a vision feature map, or a transformer block whose N positions or tokens have a fixed graph. Let H in R^(B x N x C) be the hidden tensor, L_G = U diag(mu_k) U^T the symmetric normalized graph Laplacian, and U its orthonormal eigenvector matrix. Precompute U and the eigenvalues mu_k for a fixed graph. For changing attention graphs, use a sparse Lanczos approximation or retain only the lowest K eigenvectors. For each batch index b and channel c, compute h = H[b,:,c], a = U^T h, q = sum_k ((mu_k + epsilon)^s a_k^2), ah = abs(h), aa = U^T ah, qabs = sum_k ((mu_k + epsilon)^s aa_k^2), and accumulate (q - qabs) divided by (||h||_2^2 + delta). The training loss is task_loss plus beta times the mean accumulated gap. In pseudocode: E = (mu + eps)**s; gap = 0; for b,c: h = H[b,:,c]; a = U.T @ h; q = (E * a.square()).sum(); ah = h.abs(); aa = U.T @ ah; qabs = (E * aa.square()).sum(); gap += (q-qabs)/(h.square().sum()+delta); loss = task_loss + beta*gap/(B*C). The spectral energies and their difference are computed exactly from the chosen graph approximation; the eigenpairs are estimated by eigendecomposition, Lanczos, or randomized subspace iteration. Start with Cora or PubMed and a 2-layer GCN. Compare cross-entropy alone, cross-entropy plus ordinary first-order Laplacian smoothing, and this penalty with s equal to 1.0, 1.25, and 1.49. Measure validation accuracy, calibration, edgewise feature sign-flip rate, the empirical fraction of batches with negative gaps, and training-loss variance. Success means better validation accuracy or calibration at equal parameter count, reduced sign oscillation without the accuracy loss associated with oversmoothing, and a mostly nonnegative empirical gap. A second inexpensive test can use a small ViT on CIFAR-10 with a 2D patch grid graph and evaluate robustness to pixel noise.