# Reflection-Preserving Tensor Low-Rank Layer

- ID: 149
- Canonical URL: https://synthcore.org/idea/149/reflection-preserving-tensor-low-rank-layer
- API JSON: https://synthcore.org/api/idea/149.json
- API Markdown: https://synthcore.org/api/idea/149.md
- Verification status: unverified
- Source: [arXiv:2608.24985](https://arxiv.org/abs/2608.24985)
- Category: architecture
- Solves: memory, speedup, accuracy
- ML areas: cnn, embedding, fine-tuning, inference-speedup
- Math tags: tensor-decomposition, linear-algebra, representation-theory, numerical-analysis
- Ratings: usefulness 6/10; difficulty 5/10; novelty 7/10

## Idea description

Replace a dense tensor feature basis with a truncated star_M-SVD basis computed after projection into the bilateral-reflection invariant subspace. The layer stores only the independent half of each symmetric basis slice, while reconstructing the full basis on the fly; this should reduce parameters and matrix multiplications without discarding the symmetric signal. The construction can be used as a frozen compression layer, a trainable low-rank adapter, or an initialization for a convolutional feature basis.

## Mathematical statement

The paper defines the star_M-SVD for \(\mathcal A\in\mathbb R^{m\times p\times n}\) by \(\mathcal A=\mathcal U\star_{\mathbf M}\mathcal S\star_{\mathbf M}\mathcal V^{\mathsf T}=\sum_{i=1}^{r}\mathcal U_{:,i,:}\star_{\mathbf M}\mathcal S_{i,i,:}\star_{\mathbf M}\mathcal V_{:,i,:}^{\mathsf T}\), where \(\mathcal U\) and \(\mathcal V\) are star_M-orthogonal, \(\mathcal S\) is f-diagonal, and \(r\leq\min(m,p)\) is the number of nonzero singular tubes. The transform-domain implementation forms \(\widehat{\mathcal A}=\mathcal A\times_3\mathbf M\), computes ordinary matrix SVDs \(\widehat A^{(k)}=\widehat U^{(k)}\widehat S^{(k)}(\widehat V^{(k)})^{\mathsf T}\) independently for every frontal slice \(k\), then applies \(\times_3\mathbf M^{-1}\). The Frobenius normalization is \(\lVert\mathcal B\rVert_F^2=c^{-2}\sum_k\lVert\widehat B^{(k)}\rVert_F^2\), where \(c\) is the transform normalization constant. Let \(J_m\in\mathbb R^{m\times m}\) reverse the reflected spatial coordinate, \((J_mx)_i=x_{m+1-i}\). The invariant projector is \(P_m(X)=\tfrac12(X+J_mX)\). Apply it to each transformed frontal slice before SVD. A symmetric left basis satisfies \(J_m\widehat U^{(k)}=\widehat U^{(k)}\), so only its top \(\lceil m/2\rceil\) rows need to be stored; the remaining rows are generated by reflection.

## Key formulas

- $$\mathcal{A}=\mathcal{U}\star_{\mathbf{M}}\mathcal{S}\star_{\mathbf{M}}\mathcal{V}^{\mathsf{T}}=\sum_{i=1}^{r}\mathcal{U}_{:,i,:}\star_{\mathbf{M}}\mathcal{S}_{i,i,:}\star_{\mathbf{M}}\mathcal{V}_{:,i,:}^{\mathsf{T}}.$$
- $$\widehat A^{(k)}=\widehat U^{(k)}\widehat S^{(k)}(\widehat V^{(k)})^{\mathsf T},\qquad \widehat{\mathcal A}=\mathcal A\times_3\mathbf M,\qquad \mathcal U=\widehat{\mathcal U}\times_3\mathbf M^{-1}.$$
- $$\lVert\mathcal B\rVert_F^2=\frac{1}{c^2}\sum_k\lVert\widehat B^{(k)}\rVert_F^2.$$
- $$P_m(X)=\frac12(X+J_mX),\qquad \widehat A^{(k)}_{\mathrm{sym}}=P_m(\widehat A^{(k)}),\qquad \widehat A^{(k)}_r=\widehat U^{(k)}_{:,1:r}\widehat S^{(k)}_{1:r,1:r}(\widehat V^{(k)}_{:,1:r})^{\mathsf T}.$$

## Implementation notes

Integrate this at a tensor feature bottleneck in a CNN or video encoder, before a classifier or before a costly spatial-mixing block. Let the feature tensor be \(A\in\mathbb R^{m\times p\times n}\), where \(m\) is the reflected spatial coordinate, \(p\) is channel or sample width, and \(n\) is the temporal, frequency, or feature-mode length. Choose an invertible mode transform \(M\), such as a DCT or FFT, and compute `Ahat = mode3_transform(A, M)`. For every frontal slice `k`, construct the reflection permutation matrix `J`, compute `Asym = 0.5 * (Ahat[k] + J @ Ahat[k])`, and run a rank-`r` truncated SVD `U,S,Vh = svd(Asym, rank=r)`. Enforce numerical symmetry with `U = 0.5 * (U + J @ U)` and re-orthogonalize with QR if necessary. Store only the independent upper half of `U`; in the forward pass reconstruct it as `U_full = concat(U_top, reverse(U_top))`, treating the center row separately when `m` is odd. Compute `Yhat[k] = U_full @ diag(S) @ Vh`, then apply the inverse mode transform. For a trainable low-rank adapter, initialize these factors from pretrained activations and optimize only `U_top`, `S`, and `V`; after each optimizer step project `U` with `P_m` and re-orthogonalize. The mathematical quantities are the transform-domain SVD, reflection projector, and Frobenius scaling. Estimate rank empirically using the retained-energy ratio `sum(S[:r]^2) / sum(S^2)`. Start with Fashion-MNIST, aligned CelebA faces, or a symmetric-object dataset, using ResNet-18 with a dense bottleneck as baseline. Compare accuracy, reconstruction error, forward FLOPs, and basis storage against ordinary low-rank SVD and an uncompressed layer. Success is equal accuracy at 2x-5x lower basis storage or lower latency at equal accuracy.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.