# ABP Tangential-Curvature Regularizer

- ID: 2990
- Canonical URL: https://synthcore.org/idea/2990/abp-tangential-curvature-regularizer
- API JSON: https://synthcore.org/api/idea/2990.json
- API Markdown: https://synthcore.org/api/idea/2990.md
- Verification status: unverified
- Source: [arXiv:2609.00719](https://arxiv.org/abs/2609.00719)
- Category: regularization
- Solves: generalization, stability, accuracy
- ML areas: loss, regularization, mlp, embedding
- Math tags: geometry, differential-geometry, pde, functional-analysis
- Ratings: usefulness 5/10; difficulty 6/10; novelty 6/10

## Idea description

Regularize a scalar network output so that its superlevel sets are approximately quasiconcave in input or latent space. Instead of penalizing the full Hessian, penalize positive curvature only in directions orthogonal to the output gradient, matching the paper's projected-Hessian and weighted 1-Laplacian structure.

## Mathematical statement

The paper studies the weighted 1-Laplacian equation $|\nabla u|^{1/(n-1)}\Delta_1u=f(x)$, where $\Delta_1u=\operatorname{div}(\nabla u/|\nabla u|)=|\nabla u|^{-1}\operatorname{tr}\widetilde D^2u$. More generally it uses $|\nabla u|^{(2-n)/(n-1)}F(x,\widetilde D^2u)=f(x)$, where $\widetilde D^2u$ is the Hessian restricted to directions orthogonal to $\nabla u$, and $F$ is uniformly elliptic with constants $0<\lambda\leq\Lambda$. For a twice differentiable scalar network score $u_\theta(z)$, define $g=\nabla_z u_\theta(z)$, $P_\varepsilon=I-gg^\top/(\|g\|_2^2+\varepsilon^2)$, and $H=\nabla_z^2u_\theta(z)$. The tangential Hessian is $\widetilde H=P_\varepsilon H P_\varepsilon$ and its trace is the neural analogue of $\operatorname{tr}\widetilde D^2u$. Local quasiconcavity requires $v^\top Hv\leq0$ for every $v$ with $v^\top g=0$, so a practical penalty is the squared positive part of a stochastic estimate of the weighted tangential trace. The ABP estimate is used as geometric motivation; the neural implementation is a soft empirical regularizer rather than a direct assertion of the PDE theorem.

## Key formulas

- $$|\nabla u|^{1/(n-1)}\Delta_{1}u=f(x),\qquad \Delta_{1}u=\operatorname{div}\frac{\nabla u}{|\nabla u|}=\frac{1}{|\nabla u|}\operatorname{tr}\widetilde{D}^{2}u.$$
- $$|\nabla u|^{(2-n)/(n-1)}F(x,\widetilde{D}^{2}u)=f(x),\qquad 0<\lambda\leq\Lambda.$$
- $$g=\nabla_z u_\theta(z),\quad P_\varepsilon=I-\frac{gg^\top}{\|g\|_2^2+\varepsilon^2},\quad \widetilde H=P_\varepsilon\nabla_z^2u_\theta(z)P_\varepsilon.$$
- $$\mathcal L_{\mathrm{tan}}=\frac{1}{B}\sum_{i=1}^{B}\left[\frac{\operatorname{tr}(\widetilde H_i)}{(\|g_i\|_2+\varepsilon)^{(n-2)/(n-1)}}\right]_+^2.$$

## Implementation notes

Integrate the regularizer into the loss of a scalar-output network: an energy model, binary classifier logit, scalar reward, or decoder score $u_\theta(z)$ evaluated on latent vectors or input embeddings $z$. Avoid constructing a dense Hessian. For each minibatch sample $z_i$, compute $u_i$ and its input gradient $g_i=\nabla_z u_i$. Draw $K$ Gaussian vectors $r_{ik}\sim\mathcal N(0,I_d)$, project and normalize them with $v_{ik}=P_\varepsilon r_{ik}/\|P_\varepsilon r_{ik}\|$, and compute Hessian-vector products $h_{ik}=H_i v_{ik}$ using automatic differentiation. Estimate the projected trace by $\widehat T_i=(d/K)\sum_{k=1}^K v_{ik}^{\top}h_{ik}$, where $d$ is the input or latent dimension. Add $\alpha\mathcal L_{\mathrm{tan}}$ to the task loss, using $\varepsilon=10^{-4}$ times the minibatch median gradient norm and optionally stopping gradients through $P_\varepsilon$ initially. Pseudocode: `g=grad(u,z); P=I-g@g.T/(||g||^2+eps^2); T=0; for k in K: v=normalize(P@randn(d)); T += v@HVP(u,z,v); T=(d/K)*T; Ltan += relu(T/(||g||+eps)**((d-2)/(d-1)))**2; L=Ltask+alpha*Ltan`. The projected trace and weight come from the paper's operator; the trace is estimated empirically by Hutchinson-style vectors. First test a three-layer MLP on two-moons and CIFAR-10 embeddings against the baseline and an isotropic Hessian penalty at matched optimizer steps and FLOPs. Track test accuracy, calibration, perturbation robustness, training stability, runtime, and the fraction of tangent probes having positive curvature. Success means reduced positive tangential curvature and improved accuracy or robustness at comparable training loss and overhead.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.