Quadratic-Hessian cone regularizer
Source paper: Interior $C^{2,α}$ Regularity for the Quadratic Hessian Equation arXiv:2608.29484 ⓘ · analyzed Sep 1, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
For a coordinate-based neural network u_theta(x) solving a fully nonlinear second-order PDE, replace the raw quadratic-Hessian residual with the concave, homogeneous operator G(D_x^2 u_theta)=sqrt(sigma_2(D_x^2 u_theta)). Add differentiable barriers that keep the predicted Hessian inside the positive branch Gamma_2, preventing optimization from entering regions where the PDE operator is non-elliptic.
Formulas
Mathematical statement
For a symmetric matrix A in Sym(n) with eigenvalues lambda_1,...,lambda_n, sigma_1(A)=sum_i lambda_i=tr(A) and sigma_2(A)=sum_{i<j} lambda_i lambda_j=1/2[(tr A)^2-tr(A^2)]. The admissible cone is Gamma_2={A:sigma_1(A)>0 and sigma_2(A)>0}. The paper states that G(A)=sqrt(sigma_2(A)) is elliptic, concave, and homogeneous of degree one on Gamma_2, meaning G(tA)=tG(A) for t>0. The target equation sigma_2(D^2u)=f>0 is equivalently G(D^2u)=g=sqrt(f). For a neural network, H_theta(x)=D_x^2u_theta(x), s_1(x)=tr(H_theta(x)), and s_2(x)=1/2[(tr H_theta(x))^2-tr(H_theta(x)^2)]. Use residual r_theta(x)=sqrt(max(s_2(x),epsilon))-g(x). Enforce the cone with b_1=softplus(tau(delta-s_1))/tau and b_2=softplus(tau(delta-s_2))/tau, where epsilon>0 prevents invalid square roots, delta>0 is a safety margin, and tau controls barrier sharpness. The training objective is L=E[r_theta^2+lambda_1 b_1+lambda_2 b_2]+lambda_bc L_bc. The transferable assets are the elliptic branch restriction, concavity of G, and degree-one scaling.
Implementation notes
(1) Integrate this into a coordinate MLP u_theta:R^n->R used as a PINN or implicit function model. At each interior collocation batch x_b, use automatic differentiation to compute the full input Hessian H_b=D_x^2u_theta(x_b); these are derivatives with respect to coordinates, not parameter Hessians. Compute s_1b=trace(H_b) and s_2b=0.5*(s_1b^2-sum_ij H_bij^2). Replace a raw residual sigma_2(H_b)-f(x_b) with r_b=sqrt(clamp(s_2b,min=epsilon))-sqrt(f(x_b)). Add the two cone barriers and the boundary-condition loss. (2) Pseudocode: u=net(x); H=hessian(u,x); s1=trace(H); s2=0.5*(s1*s1-(H*H).sum()); r=sqrt(clamp(s2,min=eps))-sqrt(f(x)); b1=softplus(tau*(delta-s1))/tau; b2=softplus(tau*(delta-s2))/tau; loss=mean(r*r+lam1*b1+lam2*b2)+lam_bc*boundary_loss; optimizer.step(loss). Start in float64 with epsilon=1e-8 and delta=1e-4, then anneal lambda_1 and lambda_2 from zero to their target values during the first 10-20% of training. (3) Compute sigma_2, G, and cone membership exactly from each batch Hessian. Estimate empirical cone occupancy as the fraction of points satisfying s_1>delta and s_2>delta; log the 95th percentile of ||H_b||_F and residual spatial variation. (4) First test a two-dimensional Dirichlet problem on a disk or square using a manufactured smooth solution u*. Set f=sigma_2(D^2u*)+c with c chosen so f is strictly positive. Compare the proposed loss against the raw sigma_2(H)-f loss using the same four-layer, 128-width tanh MLP, collocation points, optimizer, and number of Hessian evaluations across five seeds. Measure PDE residual, boundary error, relative C^2 error, cone-violation rate, Hessian norm, loss smoothness, and wall-clock time. Success is lower cone-violation rate and interior C^2 error, with faster residual reduction at equal Hessian-evaluation budget; failure is no improvement or excessive autodiff overhead.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.