Unverified 2026

Harnack regularization for positive feature fields

Implementation & benchmark of arXiv:2608.11107 — Harnack-type inequalities and traveling waves for non-cooperative nonlocal diffusion systems

Usefulness5/10

Difficulty3/10

Novelty6/10

Regularization Stability Generalization Cnn Ssm Graph nn Regularization Pde Dynamical systems Linear algebra Analysis

Source paper: Harnack-type inequalities and traveling waves for non-cooperative nonlocal diffusion systems arXiv:2608.11107 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Represent a feature field with positive channel amplitudes and penalize violations of the paper's system-wide relative-variation bound. Unlike per-channel total variation, the penalty constrains only aggregate channel mass, allowing channels to exchange mass through signed or non-cooperative mixing while keeping the overall representation stable. The method is most natural for intermediate CNN maps, positive SSM states, or sequence embeddings indexed by a coordinate with meaningful local neighborhoods.

Formulas

$$P_{1}p(s)\leq cp^{\prime}(s)-\mathcal{D}(s)\left(\int_{\mathbb{R}}J(s-s^{\prime})p(s^{\prime})\,ds^{\prime}-p(s)\right)\leq P_{2}p(s),$$

$$\frac{1}{|p(s)|_{1}}\left|\sum_{i=1}^{n}p_{i}^{\prime}(s)\right|\leq M,\qquad |p(s)|_1:=\sum_{i=1}^{n}|p_i(s)|,$$

$$e^{-Mr}\leq\frac{|p(\xi+s)|_{1}}{|p(\xi)|_{1}}\leq e^{Mr}\qquad (r>0),$$

$$\mathcal{L}_{\mathrm{H}}=\frac{1}{|\mathcal{P}|}\sum_{(k,\ell)\in\mathcal{P}}\left[\operatorname{ReLU}\left(\left|\log(A_{\ell}+\varepsilon)-\log(A_k+\varepsilon)\right|-M|s_\ell-s_k|\right)\right]^2,\quad A_k=\sum_{i=1}^{n}\operatorname{softplus}(h_{k,i}).$$

Mathematical statement

The paper assumes a positive vector p(s)=(p_1(s),...,p_n(s)) in (0,∞)^n, a bounded positive diagonal diffusion matrix \mathcal{D}(s)=diag(d_1(s),...,d_n(s)) with d_min≤d_i(s)≤d_max, a scalar c, kernel J, and arbitrary real matrices P_1,P_2 satisfying the componentwise inequality P_1p(s)≤cp'(s)-\mathcal{D}(s)((J*p)(s)-p(s))≤P_2p(s). The theorem gives a constant M, depending only on J,c,d_min,d_max,P_1,P_2, such that |\sum_i p_i'(s)|/|p(s)|_1≤M, where |p(s)|_1=\sum_i|p_i(s)|. It also gives e^{-Mr}≤|p(\xi+s)|_1/|p(\xi)|_1≤e^{Mr} for r>0. In a neural adaptation, define p_{k,i}=softplus(h_{k,i})>0 and A_k=\sum_i p_{k,i}; the discrete analogue is |\log(A_\ell+\varepsilon)-\log(A_k+\varepsilon)|≤M|s_\ell-s_k|. Penalizing violations of this envelope transfers the theorem's aggregate relative-variation control, although the theorem's PDE assumptions are treated as an inductive bias rather than asserted for arbitrary learned features.

Implementation notes

(1) Integration point: insert the regularizer after a selected intermediate block. For a CNN with activation tensor h[b,c,y,x], use p=softplus(h)+epsilon and treat each row, column, or flattened spatial path as the coordinate s. For a sequence model, use h[b,t,c] and s=t. Do not constrain logits directly; constrain positive intermediate features. Compute A_k=sum_c p[k,c], the aggregate mass at coordinate k.

(2) Pseudocode:

p = softplus(h) + eps
A = p.sum(dim=channel_dim)
loss_H = 0
for (k, l) in sampled_local_pairs:
    delta = abs(log(A[..., l]) - log(A[..., k]))
    limit = M * abs(s[l] - s[k])
    loss_H += relu(delta - limit)**2
loss_H /= number_of_pairs
loss = task_loss + lambda_H * loss_H

Use adjacent pairs plus random pairs within radius R. For two-dimensional maps, sample horizontal and vertical neighbors separately. The nonlocal aspect can be tested by defining a Gaussian kernel J_{kl}=exp(-(s_k-s_l)^2/(2sigma^2)) and sampling pairs with probability proportional to J_{kl}.

(3) The paper supplies the exponential envelope and its equivalent log-slope inequality. M is not analytically derived for the neural model. Start with a fixed M, or estimate M_0 during an unregularized warmup as the 95th percentile of |log(A_{k+1}+epsilon)-log(A_k+epsilon)|/Delta s, then set M=alpha M_0 for alpha in {0.5,1,2}. The theorem's aggregate control is intentional: individual channels remain free to vary.

(4) First cheap experiment: train ResNet-18 on CIFAR-10 and a 1D convolutional classifier on Speech Commands. Compare the baseline, standard activation total variation, and Harnack regularization at matched training steps and regularization sweeps. Record clean and corruption accuracy, activation outlier rate, maximum empirical log-slope, gradient-norm spikes, and training stability across five seeds. Success means fewer exploding activations and tighter aggregate ratio envelopes without clean-accuracy degradation; a stronger result would be improved corruption accuracy at equal compute.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.