# Normal-Space Quotient Encoder

- ID: 607
- Canonical URL: https://synthcore.org/idea/607/normal-space-quotient-encoder
- API JSON: https://synthcore.org/api/idea/607.json
- API Markdown: https://synthcore.org/api/idea/607.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.08939](https://arxiv.org/abs/2607.08939)
- Category: geometry
- Solves: accuracy, stability, generalization
- ML areas: embedding, cnn, graph-nn, regularization
- Math tags: differential-geometry, geometry, lie-groups, linear-algebra
- Ratings: usefulness 6/10; difficulty 5/10; novelty 5/10

## Idea description

Add a quotient-aware representation layer that separates changes caused by motion along a symmetry orbit from changes that are genuinely informative. The layer estimates orbit tangent directions from known group actions or a learned local transformation group, projects features onto the metric-orthogonal normal space, and trains the representation to be invariant along orbit directions. Unlike ordinary global pooling over augmentations, this construction is local and can adapt when orbit dimension changes at singular or highly symmetric inputs.

## Mathematical statement

The paper associates a Lie groupoid \(\mathcal{G}\rightrightarrows M\) with its Lie algebroid \(A=\ker(Ts)|_M\), where \(s\) is the source map, and anchor \(\rho=Tt|_M:A\to TM\), where \(t\) is the target map. The orbit tangent space at \(x\in M\) is \(T_x\mathcal{F}=\operatorname{im}(\rho_x)\). The extracted construction makes the bundle of Lie groups \(A\) act on an ambient tangent vector \(v\in T_xM\) by \(\xi\cdot v=\rho_x(\xi)+v\), for \(\xi\in A_x\). A Riemannian metric \(g_x\) splits the tangent space into orbit and normal components, \(T_xM=T_x\mathcal{F}\oplus T_x\mathcal{F}^{\perp}\). The neural adaptation treats the encoder Jacobian as an ambient tangent map and penalizes its orbit-tangent component. Here \(R_x\in\mathbb{R}^{n\times r}\) contains orbit tangent vectors, \(G_x\in\mathbb{R}^{n\times n}\) is a positive-definite metric matrix, \(J_f(x)\) is the Jacobian of the encoder \(f\), and \(P_{\mathrm{orb}}(x)\) is the metric-orthogonal projector onto the span of \(R_x\).

## Key formulas

- $$A\coloneqq\ker(Ts)|_{M},\qquad \rho\coloneqq Tt|_{M}\colon A\to TM.$$
- $$\xi\cdot v\coloneqq\rho(\xi)+v,\qquad \xi\in A_x,\;v\in T_xM.$$
- $$T_xM=T_x\mathcal{F}\oplus T_x\mathcal{F}^{\perp},\qquad T_x\mathcal{F}=\operatorname{im}(\rho_x).$$
- $$P_{\mathrm{orb}}(x)=R_x\big(R_x^{\mathsf T}G_xR_x+\varepsilon I_r\big)^{-1}R_x^{\mathsf T}G_x,\qquad \mathcal{L}_{\mathrm{orbit}}=\left\|P_{\mathrm{orb}}(x)J_f(x)\right\|_F^2.$$

## Implementation notes

Integrate this at the encoder output of an image, point-cloud, graph, or physical-state model. Let \(h=f_\theta(x)\in\mathbb{R}^d\) be the representation and let \(x\in\mathbb{R}^n\) be the input. Supply a matrix \(R_x\in\mathbb{R}^{n\times r}\) whose columns are orbit tangent vectors. For a known transformation family \(a\mapsto T_a x\), compute \(R_x[:,j]=\partial_{a_j}T_a x|_{a=0}\); for a finite symmetry group, use finite differences \((T_gx-x)/\delta\) for selected nonidentity transformations. Set \(G_x=I_n\) initially, or use a learned positive diagonal metric \(G_x=\operatorname{diag}(\operatorname{softplus}(q_\phi(x))+\varepsilon)\). Compute \(P_{\mathrm{orb}}=R_x(R_x^{\mathsf T}G_xR_x+\varepsilon I_r)^{-1}R_x^{\mathsf T}G_x\). Do not materialize the full encoder Jacobian: for each orbit vector \(r_j\), use automatic differentiation to obtain the Jacobian-vector product \(J_f(x)r_j\), then penalize \(\sum_j\|J_f(x)r_j\|_2^2\), which is equivalent to suppressing encoder sensitivity along the orbit. Add \(\lambda\mathcal{L}_{\mathrm{orbit}}\) to the task loss, and optionally add finite invariance loss \(\|f(T_gx)-f(x)\|_2^2\). Estimate orbit rank with an SVD of \(R_x\), dropping singular values below a threshold rather than padding with arbitrary directions. The paper provides the orbit/normal decomposition and quotient-compatibility principle; the damping \(\varepsilon\), metric parameterization, and loss weight \(\lambda\) are engineering choices. First test on rotated-MNIST or transformed CIFAR-10 using a small CNN. Compare a baseline, standard augmentation, and this regularizer at equal FLOPs. Measure clean accuracy, accuracy under unseen transformations, representation drift, and projector conditioning. Success means improved transformation robustness without clean-accuracy loss, lower drift, and stable training near rank-deficient orbit matrices.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: no_effect
- Verdict: The quotient-aware orbit regularizer was implemented and numerically verified. Projector idempotence and orbit-preservation errors were about 1e-10, the singular-origin projector was zero, and three seeds showed substantially lower representation drift with no clean-accuracy loss. This is a promising toy-level result, not yet a broad benchmark win.

### Mechanism check

- Verdict: The quotient-aware orbit regularizer was implemented and numerically verified. Projector idempotence and orbit-preservation errors were about 1e-10, the singular-origin projector was zero, and three seeds showed substantially lower representation drift with no clean-accuracy loss. This is a promising toy-level result, not yet a broad benchmark win.
- Confidence: 8/10
- Limitations: Only a small synthetic 2D radial classification task with known SO(2) actions was tested. Rotated-MNIST/CIFAR, standard augmentation, learned transformations, full FLOP matching, and broad hyperparameter sweeps were not tested.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.42255
- Paired wins: 5/8
- Benchmark verdict: no significant win

## Artifacts

- [bench_experiment.py](https://synthcore.org/code/1409/bench_experiment.py)
- [bench_report.json](https://synthcore.org/code/1409/bench_report.json)
- [experiment.py](https://synthcore.org/code/1409/experiment.py)
- [multiseed.py](https://synthcore.org/code/1409/multiseed.py)
- [multiseed_results.json](https://synthcore.org/code/1409/multiseed_results.json)
- [report.md](https://synthcore.org/code/1409/report.md)
- [report_bench_2026-09-04T204007.md](https://synthcore.org/code/1409/report_bench_2026-09-04T204007.md)
- [results.json](https://synthcore.org/code/1409/results.json)
- [Download all files as ZIP](https://synthcore.org/download/1409)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.