Symplectic Hessian curvature regularizer
Implementation & benchmark of arXiv:2608.23236 — Convex functions with symplectic Hessian
Source paper: Convex functions with symplectic Hessian arXiv:2608.23236 ⓘ · analyzed Aug 29, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace an ordinary input-convex potential with a potential whose Hessian is encouraged to be symmetric positive definite and symplectic. Add a curvature penalty based on the scalar curvature of the Hessian metric, together with a theorem-derived interior target proportional to the inverse squared distance to the domain boundary. This should suppress pathological third-derivative oscillations while preserving nonquadratic structure near boundaries.
Formulas
Mathematical statement
Theorem 1.1 states that if u: U -> R is smooth, H(x)=Hess(u)(x) is positive definite, and H(x) is in Sp(2m,R) for every x, then the scalar curvature R of the Hessian metric g=H satisfies R(x)>=0 and sqrt(R(x)) <= 3m / dist_g(x, boundary U). Here U is an open subset of R^{2m}, m is half the input dimension, g_{ij}=partial_i partial_j u, R is the scalar curvature of g, and dist_g is geodesic distance measured using g. In the full-space case U=R^{2m}, the distance-to-boundary is infinite, hence R=0 and u is quadratic. For a symmetric Hessian H, the symplectic condition H J H=J is equivalent to H being in Sp(2m,R), where J=[[0,I_m],[-I_m,0]]. For a Hessian metric define C_{ijk}=partial_i partial_j partial_k u. In affine coordinates its curvature can be computed by R_{ijkl}=1/4 g^{pq}(C_{jkp}C_{ilq}-C_{ikp}C_{jlq}), and scalar curvature by R=g^{ik}g^{jl}R_{ijkl}, up to the implementation's curvature-sign convention. The network uses the paper's exact constraint as a soft approximation: symplectic defect S=||H J H-J||_F^2, positive-definiteness barrier B=-log det(H+epsilon I), and curvature target R_target=9m^2/(d_g^2+delta), where d_g is an estimated distance to the sampled domain boundary.
Implementation notes
(1) Integrate at the potential output of an input-convex MLP used as an energy, transport potential, monotone map potential, or convex feature map. Let the input dimension be n=2m and let u_theta(x) be the scalar network output. On a small set of points x in the training batch, use automatic differentiation to compute H=Hess_x(u_theta), the third-derivative tensor C, and the task loss gradient. Do not apply this to every large batch initially; compute the geometric terms on a random geometric microbatch every K training steps.
(2) Pseudocode:
for x,y in batch:
u = net_theta(x)
H = jacobian(grad_x(u), x)
H = 0.5*(H + transpose(H))
C[i,j,k] = dH[i,j]/dx[k]
A = H + eps*I
Ginv = inverse(A)
symp = frobenius_norm(H @ J @ H - J)**2
barrier = -logdet(A)
Riem[i,j,k,l] = 0.25 * sum_pq(Ginv[p,q] *
(C[j,k,p]*C[i,l,q] - C[i,k,p]*C[j,l,q]))
R = sum_i,j,k,l Ginv[i,k]*Ginv[j,l]*Riem[i,j,k,l]
d = estimate_metric_boundary_distance(x, H, domain_box)
bound = 3*m/(d + delta)
geom = lam_s*symp + lam_b*barrier + lam_R*relu(sqrt(relu(R))-bound)**2
loss = task_loss(u,y) + geom
update(theta, grad(loss))
(3) The theorem-derived quantities are the symplectic identity H J H=J, the curvature expression, nonnegativity of R under the exact assumptions, and the 3m/dist_g bound. The implementation must estimate d_g. For a box domain, approximate the shortest distance to each face by straight-line metric lengths: d_g(x) = min_face sqrt((b_face-x)^T H(x)(b_face-x)); alternatively use 8-16 fixed line segments and trapezoidal integration of sqrt(v^T H(x(t))v). This is only a proxy for geodesic distance. Monitor the smallest eigenvalue of H, symplectic defect, signed curvature, and task loss separately. Because soft symplecticity does not imply the theorem, report the regularizer as an empirical stabilization mechanism rather than a formal guarantee.
(4) First experiment: use n=4 (m=2) input-convex MLPs on a synthetic convex regression task y=u_star(x)+noise over [-1,1]^4, with u_star a mixture of quadratic and smooth quartic terms. Compare a standard ICNN, ICNN plus only symplectic defect, and ICNN plus symplectic defect and curvature penalty at matched parameter count. Use width 128, 3 layers, AdamW, and sweep lambda_s and lambda_R. Success means lower held-out Hessian condition-number variance and third-derivative norm at equal task MSE, fewer exploding-gradient runs, and improved extrapolation just inside the domain boundary. Also test the exact full-space sanity case: with a strong symplectic penalty and no task forcing, the learned potential should approach a quadratic and R should approach zero.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.