# Refined pullback-metric Jacobian cap

- ID: 2254
- Canonical URL: https://synthcore.org/idea/2254/refined-pullback-metric-jacobian-cap
- API JSON: https://synthcore.org/api/idea/2254.json
- API Markdown: https://synthcore.org/api/idea/2254.md
- Verification status: unverified
- Source: [arXiv:2608.13682](https://arxiv.org/abs/2608.13682)
- Category: regularization
- Solves: stability, generalization
- ML areas: regularization, embedding, training-dynamics
- Math tags: differential-geometry, geometry, optimization
- Ratings: usefulness 6/10; difficulty 5/10; novelty 6/10

## Idea description

Regularize a neural encoder so its local pullback metric is bounded by the refined Schwarz-lemma constant instead of using a generic Frobenius Jacobian penalty. For an encoder into a negatively curved latent space, penalize only singular directions whose squared expansion exceeds the curvature- and dilatation-dependent threshold.

## Mathematical statement

The paper defines a V-harmonic map u by \(\tau_V(u)=\tau(u)+\mathrm{d}u(V)=0\), with Bakry–Émery Ricci tensor \(\mathrm{Ric}_V=\mathrm{Ric}-\frac12\mathcal{L}_Vg\). If the source satisfies \(\mathrm{Ric}_V\ge -Ag\) and the target has sectional curvature \(\mathrm{sec}_N\le -B<0\), then the pullback metric satisfies \(u^*h\le K_\beta g\), where \(K_\beta=\frac{A c(\beta)}{B D(\beta)}\), \(c(\beta)=\frac{\beta^2}{1+\beta^2}\), and \(D(\beta)=1-\lfloor1/c(\beta)\rfloor c(\beta)^2-(1-\lfloor1/c(\beta)\rfloor c(\beta))^2\). The inequality means \(h(\mathrm{d}u(v),\mathrm{d}u(v))\le K_\beta g(v,v)\) for every tangent vector v. For a Euclidean input metric and a latent metric matrix \(H_z\), the pullback metric is \(J_x^\top H_zJ_x\), so the neural analogue is \(\lambda_{\max}(J_x^\top H_zJ_x)\le K_\beta\). The paper also proves the looser bound \(K_\beta\le A\beta^2/B\).

## Key formulas

- $$u^{*}h\leq\frac{A c(\beta)}{B D(\beta)}g\leq\frac{A\beta^{2}}{B}g,$$
- $$c(\beta)=\frac{\beta^{2}}{1+\beta^{2}},\qquad D(\beta)=1-\left\lfloor\frac{1}{c(\beta)}\right\rfloor c(\beta)^{2}-\left(1-\left\lfloor\frac{1}{c(\beta)}\right\rfloor c(\beta)\right)^{2},$$
- $$u^{*}h(v,v)=h(\mathrm{d}u(v),\mathrm{d}u(v))\le K_{\beta}g(v,v),\qquad K_{\beta}=\frac{A c(\beta)}{B D(\beta)},$$
- $$\mathcal{L}_{\mathrm{metric}}=\frac{1}{|\mathcal{B}|}\sum_{x\in\mathcal{B}}\left[\max\left(0,\lambda_{\max}\left(J_x^{\top}H_{u(x)}J_x\right)-K_{\beta}\right)\right]^2.$$

## Implementation notes

Integrate this at the output of an encoder, projector, or hyperbolic representation network immediately after computing the latent representation z=f_theta(x). Let J_x be the Jacobian of z with respect to the input x, and let H_z be the target latent-space metric. For Euclidean latents use H_z=I; for a Poincare-ball latent with radius one use the conformal metric H_z=4(1-||z||^2)^{-2}I. Select positive scales A and B as hyperparameters, and test beta in {1, 1/sqrt(2), 1/sqrt(3), 1/sqrt(4)}. Compute c=beta^2/(1+beta^2), q=floor(1/c), D=1-q*c^2-(1-q*c)^2, and K=A*c/(B*D). Estimate the largest eigenvalue without materializing J: initialize a normalized random input-space vector v; compute Jv with a forward-mode Jacobian-vector product; multiply by H_z; compute J^T(H_z Jv) with a reverse-mode vector-Jacobian product; normalize and repeat for one or two power iterations. Estimate lambda=v^T J^T H_z Jv and add lambda_metric*relu(lambda-K)^2 to the task loss. Pseudocode: z=encoder(x); r=JVP(encoder,x,v); w=VJP(encoder,x,H_z*r); v=w/(norm(w)+epsilon); estimate=v^T*w; loss=task_loss+lambda_metric*mean(relu(estimate-K)^2); update theta. The formula-derived quantities are K, c, and D; Jacobian spectra are estimated empirically. First compare against no regularizer, a Frobenius Jacobian penalty, and the loose threshold A*beta^2/B on a small MLP for MNIST or CIFAR-10. Record clean accuracy, adversarial accuracy, top Jacobian eigenvalue, and wall-clock overhead. Success is lower local expansion and improved perturbation robustness or generalization at equal accuracy and acceptable computational cost.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.