Iterative tensor network transformations for element-wise evaluation of elementary and filtering functions

What the math gives to ML

The paper provides a constructive polynomial iteration for evaluating the discontinuous sign function element-wise while remaining in a tensor-train (TT) representation. The key transferable asset is that addition, scalar multiplication, and Hadamard products can implement nonlinear thresholding without expanding an exponentially large explicitly stored field, provided intermediate TT ranks are controlled by rounding. In neural networks, this is most promising for compressed binary gating or threshold masks over very large structured activation tensors, especially TT-compressed mixture-of-experts routing or binary-weight/activation layers. The first experiment should test whether a few cubic iterations produce useful hard gates at lower memory than densifying the TT tensor, while measuring approximation error and rank growth.