arXiv:2607.04712 2026 Architecture 1 ideas extracted · analyzed Aug 30, 2026
The paper introduces tensor subspaces invariant under permutations inside a small number of contiguous blocks, together with decompositions into complementary piecewise-symmetric and piecewise-skew components. This provides a principled way to impose local permutation symmetries on high-order neural parameters without collapsing all modes into a fully symmetric tensor. The most direct transfer is a structured tensor layer or attention interaction kernel whose block pattern controls parameter sharing, inductive bias, and computational cost.
Replace an unconstrained order-k weight tensor with a sum of components that are symmetric only within selected contiguous index blocks. This preserves interactions between blocks while tying parameters under within-block permutations, providing a tunable middle ground between a fully dense tensor and a fully symmetric tensor.