# TT Cubic Sign Router

- ID: 2435
- Canonical URL: https://synthcore.org/idea/2435/tt-cubic-sign-router
- API JSON: https://synthcore.org/api/idea/2435.json
- API Markdown: https://synthcore.org/api/idea/2435.md
- Verification status: unverified
- Source: [arXiv:2608.17135](https://arxiv.org/abs/2608.17135)
- Category: architecture
- Solves: memory, inference-speedup, scalability
- ML areas: moe-routing, quantization, memory
- Math tags: tensor-decomposition, approximation-theory, algebra, numerical-analysis
- Ratings: usefulness 6/10; difficulty 6/10; novelty 7/10

## Idea description

Replace explicit thresholding of a large structured routing-logit tensor with the paper's cubic sign iteration applied directly to its TT cores. The resulting approximate sign field can be converted into binary expert gates or binary activation masks without materializing all tensor entries, enabling routing over exponentially large Cartesian index spaces. Use TT rounding after each Hadamard product to prevent rank explosion, and compare the method against dense thresholding on smaller instances where exact outputs are available.

## Mathematical statement

The paper defines the element-wise sign function by comparing a TT tensor \(\Psi\) with the zero tensor, where \(\mathrm{Sign}(x)=1\) for \(x>0\), \(0\) for \(x=0\), and \(-1\) for \(x<0\). It computes this using \(R^{(0)}=\Psi\) and \(R^{(k+1)}=F(R^{(k)})\), with \(F(R)=-\frac{1}{2}(R^{\odot 3}-3R)\); \(R^{\odot 3}\) means element-wise cubing, not a matrix or tensor contraction. The iteration converges element-wise to \(\mathrm{Sign}(\Psi)\) when every entry satisfies \(|\Psi^{[j]}|<\sqrt{3}\), where \(j\) indexes tensor entries. For a neural router, let \(L\) be a TT-compressed tensor of routing logits, let \(a>\max_j|L^{[j]}|/\sqrt{3}\) be a safety scale, and initialize \(R^{(0)}=L/a\). Then \(R^{(K)}\) approximates \(\mathrm{Sign}(L)\), and a binary gate is \(G=(1+R^{(K)})/2\). Every addition and scalar multiplication is performed core-wise; every Hadamard cube is implemented by TT core products followed by TT-rounding to tolerance \(\varepsilon\).

## Key formulas

- $$R^{(k+1)}=F\left(R^{(k)}\right),\qquad R^{(0)}=\Psi,$$
- $$F(R^{(k)})=-\frac{1}{2}\left((R^{(k)})^{\odot 3}-3R^{(k)}\right),$$
- $$|\Psi^{[j]}|<\sqrt{3}\ \ (\forall j),\qquad \mathrm{Sign}(x)=\begin{cases}1,&x>0\\0,&x=0\\-1,&x<0\end{cases},$$
- $$R^{(0)}=\frac{L}{a},\qquad R^{(k+1)}=-\frac12\left((R^{(k)})^{\odot3}-3R^{(k)}\right),\qquad G=\frac{1+R^{(K)}}{2}.$$

## Implementation notes

1. Integration point: use this in a TT-compressed MoE router or binary activation layer. Let the router produce a structured logit tensor \(L\) over a Cartesian product of token, spatial, feature, or expert-index modes, stored as TT cores rather than a dense tensor. The gate should be computed at inference time first; training can use a straight-through estimator or backpropagate through the polynomial iterations. 2. Pseudocode: estimate \(a=(1+\delta)\max_j|L^{[j]}|/\sqrt{3}\), using exact dense max only for small validation tensors or a TT max/upper bound for production; set \(R=L/a\). For \(k=1,\ldots,K\), compute \(Q=R\odot R\), round TT\((Q)\) to relative tolerance \(\varepsilon\), compute \(C=Q\odot R\), round TT\((C)\), and set \(R=1.5R-0.5C\). Return \(G=(1+R)/2\), optionally thresholding \(G>0.5\) to obtain a binary route. 3. Mathematical versus empirical quantities: the cubic map and convergence domain come from the paper; \(K\), TT-rounding tolerance \(\varepsilon\), safety factor \(\delta\), and whether to use straight-through gradients must be tuned. Track TT ranks after each product, maximum gate error, and the fraction of entries near zero. If logits are too close to zero, add a learned or fixed margin \(m\) and apply the iteration to \(L-m\), or use a soft surrogate during early training. 4. First experiment: construct a small TT router for a 4D or 6D synthetic activation tensor and a 2-layer MLP-MoE on MNIST or CIFAR-10. Compare dense sign/top-1 routing, sigmoid routing, and TT cubic routing for \(K\in\{2,4,6,8\}\), several \(\varepsilon\), and equal routing decisions. Success means at least 4x lower peak activation memory or substantially larger addressable routing tensors at similar accuracy, with TT gate error below 1% on non-margin entries and lower inference cost than densifying the tensor.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.