Spectral aspects of random heavy-tailed tensors
arXiv:2607.24695
2026
Architecture
1 ideas extracted · analyzed Aug 31, 2026
What the math gives to ML
The paper identifies a sparse heavy-Wigner tensor regime in which every tensor entry has moments of order N^{-(p-1)}, while contractions retain an O(1/N) matrix variance and locally behave like hypertrees. This suggests a principled alternative to dense Gaussian tensor initialization: place weights on a sparse random hypergraph and choose the nonzero mark distribution so that forward variance and contraction moments are explicitly controlled. The most promising neural transfer is a tensorized layer with tunable hypergraph connectivity, using contraction-spectrum measurements to detect exploding or vanishing multilinear dynamics.
Ideas from this paper
Unverified
2026
Replace a dense order-p tensorized neural layer with a sparse random hypergraph tensor whose support probability is alpha = c/N^{p-1}. Assign independent zero-mean marks to active hyperedges, so every tensor moment scales as N^{-(p-1)} while the total variance entering each output remains controlled. The resulting locally tree-like computation graph should reduce parameters and multiply-adds without immediately destroying signal propagation.
Useful6/10
Difficulty5/10
Novelty6/10