# Annealed Infinity-Harmonic Dual Head

- ID: 489
- Canonical URL: https://synthcore.org/idea/489/annealed-infinity-harmonic-dual-head
- API JSON: https://synthcore.org/api/idea/489.json
- API Markdown: https://synthcore.org/api/idea/489.md
- Verification status: unverified
- Source: [arXiv:2607.06698](https://arxiv.org/abs/2607.06698)
- Category: regularization
- Solves: stability, generalization, accuracy
- ML areas: loss, regularization, embedding
- Math tags: pde, optimization, differential-geometry, geometry
- Ratings: usefulness 5/10; difficulty 5/10; novelty 6/10

## Idea description

Add a two-channel geometric head producing scalar fields u(x) and v(x) on a two-dimensional input or latent coordinate domain. Train it initially with a moderate p-harmonic duality constraint, then anneal p upward so u approaches an infinity-harmonic field while v remains its rotated-gradient dual; this penalizes isolated steep gradient spikes and promotes smooth, coherent level sets.

## Mathematical statement

The paper defines a p-harmonic function u by \(\Delta_p u=\operatorname{div}(|\nabla u|^{p-2}\nabla u)=0\), equivalently as a local minimizer of the Dirichlet quantity \(\|\nabla u\|_{L^p}\). In two dimensions, for conjugate exponents \(p,q\) satisfying \(1/p+1/q=1\), its dual q-harmonic field v obeys \(\nabla v=|\nabla u|^{p-2}\nabla^\perp u\), where \(\nabla^\perp u=(-\partial_yu,\partial_xu)\) is the counterclockwise 90-degree rotation of \(\nabla u\). The p-to-infinity limit is the infinity-Laplacian equation \(\Delta_\infty u:=\nabla^2u(\nabla u,\nabla u)=0\), which is also characterized by absolute minimization of \(\|\nabla u\|_{L^\infty}\): among functions with the same boundary values, no local perturbation lowers the maximum gradient norm. Implement the duality as a residual loss, using p as an annealed finite surrogate for the nonsmooth \(L^\infty\) objective. The Hessian \(\nabla^2u\) is computed by automatic differentiation; \(\varepsilon\) stabilizes powers near critical points.

## Key formulas

- $$\Delta_p u:=\operatorname{div}\big(|\nabla u|^{p-2}\nabla u\big)=0,$$
- $$\nabla v=|\nabla u|^{p-2}\nabla^{\perp}u,\qquad \nabla u=-|\nabla v|^{q-2}\nabla^{\perp}v,\qquad \frac1p+\frac1q=1,$$
- $$\Delta_\infty u=\nabla^2u(\nabla u,\nabla u)=0,$$
- $$\mathcal L_{\mathrm{dual}}=\frac1{|B|}\sum_{x\in B}\left\|\nabla v_\theta(x)-\big(\|\nabla u_\theta(x)\|_2^2+\varepsilon^2\big)^{(p-2)/2}R\nabla u_\theta(x)\right\|_2^2+\lambda\,\frac1{|B|}\sum_{x\in B}\left(\frac{\nabla u_\theta(x)^\top\nabla^2u_\theta(x)\nabla u_\theta(x)}{\|\nabla u_\theta(x)\|_2^2+\varepsilon^2}\right)^2,$$

## Implementation notes

Integrate this as a regularizer on a two-output MLP head applied to two-dimensional coordinates x=(x_1,x_2), or to a selected two-dimensional latent chart. The first output is u_theta(x), and the second is v_theta(x); retain the ordinary task loss if the head is part of a larger model. Define R(a,b)=(-b,a), and compute first and second spatial derivatives with automatic differentiation rather than parameter derivatives. At each training stage choose p in [2,16], q=p/(p-1), and epsilon=1e-4 times the median batch gradient norm. Pseudocode: sample coordinates x; evaluate u and v; compute gu=grad(u,x), gv=grad(v,x), and Hu=hessian(u,x); compute dual=(||gu||^2+eps^2)^((p-2)/2)*R(gu); compute infinity=(gu^T Hu gu)/(||gu||^2+eps^2); set loss=task_loss + lambda_dual*mean(||gv-dual||^2) + lambda_inf*mean(infinity^2); update network parameters. Start with p=2 for 10–20% of training, then increase geometrically to 4, 8, and 16 while reducing lambda_dual if it dominates the task loss. No theorem constants need to be estimated: the required quantities are gradients, Hessians, and residuals. For the first experiment, use a coordinate MLP fitting a noisy 2D signed-distance or image-regression field. Compare a standard MLP, explicit gradient-norm regularization, and the proposed dual head at equal parameter count. Measure validation error, worst-case local gradient spikes, infinity-Laplacian residual, Hessian residual, and robustness to coordinate perturbations. Success means lower 99th-percentile gradient magnitude and better perturbed-input error without slower convergence or more than 2x training cost.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.