Normal-Space Quotient Encoder

Integrate this at the encoder output of an image, point-cloud, graph, or physical-state model. Let \(h=f_\theta(x)\in\mathbb{R}^d\) be the representation and let \(x\in\mathbb{R}^n\) be the input. Supply a matrix \(R_x\in\mathbb{R}^{n\times r}\) whose columns are orbit tangent vectors. For a known transformation family \(a\mapsto T_a x\), compute \(R_x[:,j]=\partial_{a_j}T_a x|_{a=0}\); for a finite symmetry group, use finite differences \((T_gx-x)/\delta\) for selected nonidentity transformations. Set \(G_x=I_n\) initially, or use a learned positive diagonal metric \(G_x=\operatorname{diag}(\operatorname{softplus}(q_\phi(x))+\varepsilon)\). Compute \(P_{\mathrm{orb}}=R_x(R_x^{\mathsf T}G_xR_x+\varepsilon I_r)^{-1}R_x^{\mathsf T}G_x\). Do not materialize the full encoder Jacobian: for each orbit vector \(r_j\), use automatic differentiation to obtain the Jacobian-vector product \(J_f(x)r_j\), then penalize \(\sum_j\|J_f(x)r_j\|_2^2\), which is equivalent to suppressing encoder sensitivity along the orbit. Add \(\lambda\mathcal{L}_{\mathrm{orbit}}\) to the task loss, and optionally add finite invariance loss \(\|f(T_gx)-f(x)\|_2^2\). Estimate orbit rank with an SVD of \(R_x\), dropping singular values below a threshold rather than padding with arbitrary directions. The paper provides the orbit/normal decomposition and quotient-compatibility principle; the damping \(\varepsilon\), metric parameterization, and loss weight \(\lambda\) are engineering choices. First test on rotated-MNIST or transformed CIFAR-10 using a small CNN. Compare a baseline, standard augmentation, and this regularizer at equal FLOPs. Measure clean accuracy, accuracy under unseen transformations, representation drift, and projector conditioning. Success means improved transformation robustness without clean-accuracy loss, lower drift, and stable training near rank-deficient orbit matrices.