ABP Tangential-Curvature Regularizer
Source paper: On the Aleksandrov--Bakelman--Pucci estimates for the weighted $1$-Laplacian arXiv:2609.00719 ⓘ · analyzed Sep 2, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Regularize a scalar network output so that its superlevel sets are approximately quasiconcave in input or latent space. Instead of penalizing the full Hessian, penalize positive curvature only in directions orthogonal to the output gradient, matching the paper's projected-Hessian and weighted 1-Laplacian structure.
Formulas
Mathematical statement
The paper studies the weighted 1-Laplacian equation $|\nabla u|^{1/(n-1)}\Delta_1u=f(x)$, where $\Delta_1u=\operatorname{div}(\nabla u/|\nabla u|)=|\nabla u|^{-1}\operatorname{tr}\widetilde D^2u$. More generally it uses $|\nabla u|^{(2-n)/(n-1)}F(x,\widetilde D^2u)=f(x)$, where $\widetilde D^2u$ is the Hessian restricted to directions orthogonal to $\nabla u$, and $F$ is uniformly elliptic with constants $0<\lambda\leq\Lambda$. For a twice differentiable scalar network score $u_\theta(z)$, define $g=\nabla_z u_\theta(z)$, $P_\varepsilon=I-gg^\top/(\|g\|_2^2+\varepsilon^2)$, and $H=\nabla_z^2u_\theta(z)$. The tangential Hessian is $\widetilde H=P_\varepsilon H P_\varepsilon$ and its trace is the neural analogue of $\operatorname{tr}\widetilde D^2u$. Local quasiconcavity requires $v^\top Hv\leq0$ for every $v$ with $v^\top g=0$, so a practical penalty is the squared positive part of a stochastic estimate of the weighted tangential trace. The ABP estimate is used as geometric motivation; the neural implementation is a soft empirical regularizer rather than a direct assertion of the PDE theorem.
Implementation notes
Integrate the regularizer into the loss of a scalar-output network: an energy model, binary classifier logit, scalar reward, or decoder score $u_\theta(z)$ evaluated on latent vectors or input embeddings $z$. Avoid constructing a dense Hessian. For each minibatch sample $z_i$, compute $u_i$ and its input gradient $g_i=\nabla_z u_i$. Draw $K$ Gaussian vectors $r_{ik}\sim\mathcal N(0,I_d)$, project and normalize them with $v_{ik}=P_\varepsilon r_{ik}/\|P_\varepsilon r_{ik}\|$, and compute Hessian-vector products $h_{ik}=H_i v_{ik}$ using automatic differentiation. Estimate the projected trace by $\widehat T_i=(d/K)\sum_{k=1}^K v_{ik}^{\top}h_{ik}$, where $d$ is the input or latent dimension. Add $\alpha\mathcal L_{\mathrm{tan}}$ to the task loss, using $\varepsilon=10^{-4}$ times the minibatch median gradient norm and optionally stopping gradients through $P_\varepsilon$ initially. Pseudocode: g=grad(u,z); [email protected]/(||g||^2+eps^2); T=0; for k in K: v=normalize(P@randn(d)); T += v@HVP(u,z,v); T=(d/K)*T; Ltan += relu(T/(||g||+eps)((d-2)/(d-1)))2; L=Ltask+alpha*Ltan. The projected trace and weight come from the paper's operator; the trace is estimated empirically by Hutchinson-style vectors. First test a three-layer MLP on two-moons and CIFAR-10 embeddings against the baseline and an isotropic Hessian penalty at matched optimizer steps and FLOPs. Track test accuracy, calibration, perturbation robustness, training stability, runtime, and the fraction of tangent probes having positive curvature. Success means reduced positive tangential curvature and improved accuracy or robustness at comparable training loss and overhead.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.