Geodesic Low-Rank Latent Bottleneck
Implementation & benchmark of arXiv:2608.06971 — Mixture of Geodesic Factor Analyzers on Riemannian Homogeneous Spaces
Source paper: Mixture of Geodesic Factor Analyzers on Riemannian Homogeneous Spaces arXiv:2608.06971 ⓘ · analyzed Aug 31, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace a Euclidean low-rank latent decoder with a geodesic factor decoder on a Riemannian manifold. A learned location α provides the component center, a small set of tangent loading vectors V captures anisotropic variation, and latent coefficients z generate curved manifold-valued features through the exponential map. Multiple such decoders can form a mixture-of-geodesic-experts layer for multimodal representations.
Formulas
Mathematical statement
The paper's geodesic factor model assumes an observation x in a Riemannian manifold M has the form x = Exp(Exp(α,Vz),ε). Here α ∈ M is a base/location point, V=(v₁,…,v_q) consists of q tangent loading vectors v_i ∈ T_αM, z ∼ N(0,I_q) is a q-dimensional standard Gaussian factor, and Exp_p(u) maps a tangent vector u ∈ T_pM to the endpoint of the geodesic starting at p with initial velocity u. The inner map Exp_α(Vz) creates a low-dimensional anisotropic factor displacement, while the outer map Exp_p(ε) adds residual tangent noise ε at the displaced point p. For neural use, predict z from an input h and use a deterministic residual ε=R(h) or Gaussian ε∼N(0,σ²I) in the appropriate tangent space. Since Vz=∑ᵢzᵢvᵢ, rank q controls parameter count and anisotropic directions are learned rather than imposed by geodesic distance alone.
Implementation notes
(1) Integration point: add this module after a neural encoder produces h∈R^d and before a manifold-valued decoder, classifier, retrieval head, or contrastive loss. Start with the unit sphere S^{m−1}, where α is normalized, v_i are projected into T_αS^{m−1}, and Exp_α(u)=cos(||u||)α+sin(||u||)u/||u||. The same interface can later use SPD exponential maps. Use q much smaller than the tangent dimension.
(2) Pseudocode:
h = encoder(x)
z = MLP_z(h) # q coefficients
V = tangent_project(V_raw, alpha) # columns v_i in T_alpha M
u = V @ z
p = exp_map(alpha, u) # paper's inner Exp(alpha,Vz)
r = residual_head(h, p) # tangent vector at p
r = clip_norm(r, r_max)
x_hat = exp_map(p, r) # adapted outer Exp(p,epsilon)
loss = task_loss(x_hat, target) + beta*||z||^2 + lambda*||V.T@V-I||_F^2
For a mixture, maintain K sets (α_k,V_k), compute gating probabilities π_k=softmax(g(h)), decode every expert, and use either the weighted Fréchet reconstruction loss ∑_kπ_k d(x,x_hat_k)^2 or a soft minimum.
(3) Compute exponential maps and geodesic distances from a tested manifold library or closed-form sphere routines. Estimate no theorem constants; empirically monitor tangent norms, map numerical failures, reconstruction distance, and effective rank of V. Re-orthogonalize V in tangent coordinates or retain the Frobenius penalty. Clip ||Vz|| and ||r|| to stay below the injectivity-radius regime where the chosen map is well-conditioned.
(4) First experiment: train a small spherical autoencoder on normalized CIFAR-10 or ImageNet embeddings, comparing a Euclidean linear bottleneck, an isotropic spherical latent, and this q=4/8 geodesic factor bottleneck at equal latent width and parameter count. Measure validation geodesic reconstruction error, downstream linear-probe accuracy, robustness to Gaussian input perturbations, and training failures. Success means lower geodesic error and better accuracy at the same bottleneck size, especially when data occupy anisotropic curved clusters.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.