Geodesic curvature regularization for hidden trajectories

Integrate the regularizer at the residual-stream level of a small Transformer or at successive time steps of a diffusion U-Net. For each selected hidden trajectory z_0,…,z_T∈R^D, use a fixed random orthogonal projection P to three dimensions and normalize x_i=(Pz_i)/(|Pz_i|_2+ε), so x_i lies approximately on S^2. For every adjacent pair compute d_i=acos(clip(x_i·x_{i+1},−1+ε,1−ε)). Compute the spherical logarithm Log_x(y)=d(y−(x·y)x)/(sqrt(1−(x·y)^2)+ε), then set u_i=Log_{x_i}(x_{i+1})/(d_i+ε). Parallel-transport u_{i−1} from x_{i−1} to x_i using the standard closed-form sphere transport; for an MVP, project it onto the tangent plane at x_i and renormalize, yielding q_{i−1}. Add λ_geo R_geo to the task loss, with p=2 initially and masks for padding or inactive positions. The paper supplies the continuum curvature target and the p>1 energy; the sphere embedding and finite-dimensional projection are engineering adaptations. First test a 6-layer ViT on CIFAR-10, comparing cross-entropy plus this penalty with cross-entropy plus λΣ_i||z_{i+1}−z_i||². Sweep λ_geo over {1e−5,1e−4,1e−3}; record validation accuracy, gradient-norm variance, angle outliers, and loss at equal FLOPs. Success means fewer sharp tangent-angle spikes and equal or better accuracy. In diffusion, also measure FID at fixed denoising steps and stability when reducing the number of sampling steps.