# Geodesic curvature regularization for hidden trajectories

- ID: 1213
- Canonical URL: https://synthcore.org/idea/1213/geodesic-curvature-regularization-for-hidden-trajectories
- API JSON: https://synthcore.org/api/idea/1213.json
- API Markdown: https://synthcore.org/api/idea/1213.md
- Verification status: unverified
- Source: [arXiv:2607.21056](https://arxiv.org/abs/2607.21056)
- Category: regularization
- Solves: stability, generalization
- ML areas: regularization, transformer, diffusion
- Math tags: geometry, differential-geometry, calculus-of-variations, approximation-theory
- Ratings: usefulness 5/10; difficulty 6/10; novelty 7/10

## Idea description

Represent a sequence of hidden states as points on a Riemannian sphere and penalize discrete geodesic curvature rather than merely penalizing adjacent-state differences. The regularizer discourages sharp bends in representation trajectories while remaining comparatively insensitive to uniform traversal speed, making it suitable for transformer depth trajectories or diffusion denoising paths.

## Mathematical statement

The paper's smooth-curve energy is the total p-curvature E_p(c)=∫_c |k_S(c)|_g^p ds for exponent p>1, where c is a curve on the Riemannian surface S, s is arc length, k_S(c) is its geodesic curvature vector, and |·|_g is the metric norm. Its polygonal construction uses two consecutive geodesic edges with tangent vectors v_1 and v_2 and defines the angle α between them by cos α=⟨v_1,v_2⟩_g/(|v_1|_g|v_2|_g), with turning angle θ=π−α under the paper's orientation convention. The main result states that the relaxed polygonal energy is finite exactly when the arc-length parameterization belongs to W^{2,p}, and then equals E_p(c). For a spherical neural trajectory x_0,…,x_T∈S^2, define adjacent geodesic lengths d_i=arccos(clamp(x_i^T x_{i+1},−1+ε,1−ε)), local step h_i=(d_{i−1}+d_i)/2, and unit tangent u_i=Log_{x_i}(x_{i+1})/(d_i+ε). Parallel-transport u_{i−1} to x_i to obtain q_{i−1}. The discrete curvature angle is θ_i=arccos(clamp(q_{i−1}^T u_i,−1+ε,1−ε)), and a discrete analogue of ∫κ^p ds is Σ_i θ_i^p h_i^{1−p}.

## Key formulas

- $$E_p(c)=\int_c |k_S(c)|_g^p\,ds,\qquad p>1.$$
- $$\cos\alpha=\frac{\langle v_1,v_2\rangle_g}{|v_1|_g|v_2|_g},\qquad \theta=\pi-\alpha.$$
- $$d_i=\arccos\!\bigl(\operatorname{clip}(x_i^\top x_{i+1},-1+\varepsilon,1-\varepsilon)\bigr),\qquad h_i=\frac{d_{i-1}+d_i}{2}.$$
- $$\mathcal R_{\mathrm{geo}}(x_{0:T})=\frac{1}{T-1}\sum_{i=1}^{T-1}\theta_i^p h_i^{1-p},\qquad \theta_i=\arccos\!\bigl(\operatorname{clip}(q_{i-1}^{\top}u_i,-1+\varepsilon,1-\varepsilon)\bigr).$$

## Implementation notes

Integrate the regularizer at the residual-stream level of a small Transformer or at successive time steps of a diffusion U-Net. For each selected hidden trajectory z_0,…,z_T∈R^D, use a fixed random orthogonal projection P to three dimensions and normalize x_i=(Pz_i)/(|Pz_i|_2+ε), so x_i lies approximately on S^2. For every adjacent pair compute d_i=acos(clip(x_i·x_{i+1},−1+ε,1−ε)). Compute the spherical logarithm Log_x(y)=d(y−(x·y)x)/(sqrt(1−(x·y)^2)+ε), then set u_i=Log_{x_i}(x_{i+1})/(d_i+ε). Parallel-transport u_{i−1} from x_{i−1} to x_i using the standard closed-form sphere transport; for an MVP, project it onto the tangent plane at x_i and renormalize, yielding q_{i−1}. Add λ_geo R_geo to the task loss, with p=2 initially and masks for padding or inactive positions. The paper supplies the continuum curvature target and the p>1 energy; the sphere embedding and finite-dimensional projection are engineering adaptations. First test a 6-layer ViT on CIFAR-10, comparing cross-entropy plus this penalty with cross-entropy plus λΣ_i||z_{i+1}−z_i||². Sweep λ_geo over {1e−5,1e−4,1e−3}; record validation accuracy, gradient-norm variance, angle outliers, and loss at equal FLOPs. Success means fewer sharp tangent-angle spikes and equal or better accuracy. In diffusion, also measure FID at fixed denoising steps and stability when reducing the number of sampling steps.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.