Unverified 2026

Reverse-HLS Feature Dispersion Regularizer

Implementation & benchmark of arXiv:2607.05928 — Reversed inequality of the Herbst-type and the related Euler-Lagrange system

Usefulness5/10

Difficulty4/10

Novelty7/10

Regularization Generalization Stability Accuracy Regularization Cnn Vae Training dynamics Functional analysis Measure theory Optimization Pde

Source paper: Reversed inequality of the Herbst-type and the related Euler-Lagrange system arXiv:2607.05928 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Apply the paper's reversed weighted interaction inequality to two nonnegative feature maps generated from different augmentations or network branches. Maximizing the normalized nonlocal interaction should discourage collapsed or overly concentrated spatial representations while remaining invariant to overall feature amplitude.

Formulas

$$\left|\int_{\mathbb{R}^{n}}\int_{\mathbb{R}^{n}}|x-y|^{\alpha/q^{\prime}-n}|y|^{\alpha/q^{\prime}}g(x)h(y)\,dxdy\right|\geq C_{n,\alpha,p,q^{\prime}}\|g\|_{L^{q^{\prime}}(\mathbb{R}^{n})}\|h\|_{L^{p}(\mathbb{R}^{n})},\quad 0<p,q^{\prime}<1,\ \alpha>n,\ \frac{1}{p}+\frac{1}{q^{\prime}}-\frac{2\alpha}{q^{\prime}n}=1.$$

$$I(g,h)=\sum_{i,j}w_iw_jK_{ij}g_i h_j,\qquad K_{ij}=\left(\|x_i-x_j\|_2+\delta\right)^{\alpha/q^{\prime}-n}\left(\|x_j\|_2+\delta\right)^{\alpha/q^{\prime}}.$$

$$Q_r(z)=\left(\sum_iw_i z_i^r+\varepsilon\right)^{1/r},\qquad R(g,h)=\frac{I(g,h)}{Q_{q^{\prime}}(g)Q_p(h)+\varepsilon},\qquad L_{\mathrm{rev}}=-\log(R+\varepsilon).$$

$$u(x)=\int_{\mathbb{R}^{n}}|x-y|^{\beta-n}v^{-p_2}(y)|y|^{\beta}dy,\qquad v(x)=\int_{\mathbb{R}^{n}}|x-y|^{\beta-n}u^{-p_1}(y)|x|^{\beta}dy.$$

Mathematical statement

The paper proves that for n >= 1, 0 < p,q' < 1, alpha > n, and 1/p + 1/q' - 2 alpha/(q'n) = 1, nonnegative functions g in L^{q'} and h in L^p satisfy |integral integral |x-y|^{alpha/q'-n}|y|^{alpha/q'}g(x)h(y) dx dy| >= C_{n,alpha,p,q'} ||g||_{L^{q'}} ||h||_{L^p}. Here x,y are coordinates in R^n, g and h are nonnegative feature densities, C_{n,alpha,p,q'} is a positive constant, and ||z||_{L^r} = (integral z^r)^{1/r} is a quasi-norm because r < 1. On a grid with points x_i and quadrature weights w_i, use I(g,h) = sum_{i,j} w_i w_j K_{ij} g_i h_j, where K_{ij} = (||x_i-x_j||_2 + delta)^{alpha/q'-n}(||x_j||_2 + delta)^{alpha/q'}. Define Q_r(z) = (sum_i w_i z_i^r + epsilon)^{1/r}. The transferable scale-normalized quantity is R = I(g,h)/(Q_{q'}(g)Q_p(h)+epsilon), and minimize L_rev = -log(R+epsilon). The paper's Euler-Lagrange extremals obey u(x) = integral |x-y|^{beta-n}v(y)^{-p_2}|y|^beta dy and v(x) = integral |x-y|^{beta-n}u(y)^{-p_1}|x|^beta dy; this supports using separate positive feature branches, although the first implementation need not solve this system.

Implementation notes

(1) Exact integration point: attach the loss to a low-resolution nonnegative feature tensor, such as the 14 by 14 pre-pooling feature map of a ResNet or an encoder feature map in a VAE. Use two independently augmented views of the same input. Convert signed activations a and b into g = softplus(a) and h = softplus(b), then flatten the spatial grid into N points x_i in [-1,1]^2, so n = 2 and w_i = 1/N. (2) Pseudocode: precompute K[i,j] = (norm(x[i]-x[j]) + delta)^(alpha/qprime - n) * (norm(x[j]) + delta)^(alpha/qprime); for each batch, compute g = softplus(feature_view_1), h = softplus(feature_view_2); I = sum_i,j w_i*w_j*K[i,j]*g[i]*h[j]; Qg = (sum_i w_i*g[i]^qprime + epsilon)^(1/qprime); Qh = (sum_i w_i*h[i]^p + epsilon)^(1/p); R = I/(Qg*Qh + epsilon); loss = task_loss - lambda*log(R + epsilon); backpropagate loss. (3) The kernel, quasi-norms, and ratio are computed from the paper's mathematics. The theorem constant C is not needed and should not be estimated. Use delta = 1e-3 and epsilon = 1e-8. Choose p, qprime, and alpha satisfying the displayed constraint and alpha > 2; for example, p = 0.4 and qprime = 0.4 gives alpha = 0.8, which violates alpha > 2, so do not use it. Instead enumerate admissible triples numerically, such as p = 0.8, qprime = 0.3, which gives alpha approximately 2.16. Verify the constraint and alpha > 2 in code before training. Clip kernel values to the 99.9th percentile if gradients become unstable. (4) First experiment: train ResNet-18 on CIFAR-10 with standard random crops and flips, comparing cross-entropy alone, cross-entropy plus this regularizer, and cross-entropy plus a standard variance regularizer. Start with lambda in {1e-4, 1e-3, 1e-2}. Record validation accuracy, feature-map effective support exp(H(g))/N where H(g) is entropy of normalized g, cross-view cosine similarity, gradient-norm variance, and training FLOPs. A positive result is higher accuracy or corruption robustness at equal FLOPs, increased effective support without activation explosion, and a gain that disappears when K is replaced by a randomized matrix.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.