Hypertree-Sparse Tensor Initialization

Integrate this at the parameter and forward-kernel level of an order-p tensorized MLP or multilinear feature layer. Let every mode have width N and choose a target active degree c. Sample one Bernoulli support mask B with probability c/N^{p-1}, then store only active index tuples and their trainable marks Z. For a first implementation, use ordered tuples rather than enforcing tensor symmetry; a second experiment can canonicalize tuples under index permutations and share one parameter across each orbit. Initialize active marks with Z drawn from a zero-mean unit-variance Gaussian, so tau^2 = 1. For input vectors x^(2), ..., x^(p), compute only active products and accumulate them into the output: for each active tuple a = (i_1, ..., i_p), set s[i_1] += Z[a] times the product of x^(r)[i_r] for r = 2,...,p; return y = g s / sqrt(c tau^2), optionally followed by bias and activation. The paper supplies the moment scaling and contraction variance; estimate the actual contraction spectral radius with 5 to 10 power iterations on random probe vectors, and estimate activation and gradient variances empirically. First test a 4-layer tensorized MLP on CIFAR-10 or a small transformer feed-forward block, comparing dense Xavier initialization and magnitude-pruned dense weights at equal stored parameters and equal measured FLOPs. Record forward variance, gradient variance, contraction spectral radius, loss versus optimizer steps, wall-clock time, and final accuracy. The construction is successful if it gives at least 2 times fewer stored parameters or multiply-adds with no more than a 1 percentage point accuracy loss, while reducing exploding-gradient or dead-activation failures relative to an unscaled sparse baseline.