Unverified
2026
Replace a dense Gaussian or learned projection from dimension N to m with a normalized partial circulant projection generated by a single Gaussian vector. For K-sparse hidden states, the restricted-isometry guarantee predicts approximate norm preservation while reducing stored projection parameters from O(mN) to O(N). The projection can be evaluated with an FFT and should be combined with explicit top-k gating so that the sparse-input assumption is enforced.
Useful6/10
Difficulty4/10
Novelty5/10
Unverified
2026
Parameterize a large linear layer as a sum of binary tensor products, W = Σ_l A_l ⊗ B_l, and regularize a factor-level upper bound on its top-k singular-value sums. The bound controls all Ky Fan norms of W while requiring SVDs only of the small factors, making it suitable for tensorized MLP or attention projections.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace a dense order-p tensorized neural layer with a sparse random hypergraph tensor whose support probability is alpha = c/N^{p-1}. Assign independent zero-mean marks to active hyperedges, so every tensor moment scales as N^{-(p-1)} while the total variance entering each output remains controlled. The resulting locally tree-like computation graph should reduce parameters and multiply-adds without immediately destroying signal propagation.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Use the paper's correspondence between KAN splines and finite-element or isogeometric shape functions to build coordinate-separable tensor-product trial layers. Replace additive coordinate aggregation with a multiplicative contraction of one-dimensional spline expansions, yielding an explicit tensor-product basis without storing a dense multidimensional grid.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace a dense tensor-product positional encoding or first MLP layer with a hierarchical sparse-grid B-spline feature map. The network evaluates only localized basis functions indexed by multi-levels with bounded total level, reducing feature count while retaining high-order approximation for functions with mixed derivative regularity. The basis can initially be fixed and later fine-tuned jointly with the downstream network.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Compress a dense neural-network weight tensor with mode-wise selected fibers and a small sampled core, then convert the result into a Tucker layer for efficient execution. Select indices greedily using reconstruction residuals rather than random sampling, and fine-tune the compressed model to recover task accuracy.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Replace a dense block of a large neural-network weight matrix with an adaptively constructed cross approximation, but prevent premature termination using residual checks on O(m+n) additional diagonal-like entries. Accept a rank only after the probe residual has remained below tolerance for several consecutive iterations; otherwise continue adding pivots.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace a $K$-class softmax with $N$ binary hyperplane heads, where each class is represented by the signs of its projections onto fixed random directions. Train the embedding to reproduce these codewords and decode by nearest Hamming codeword. The paper's guarantee suggests that $N\approx 2\log_2 K+\log_2(1/\delta)$ can separate all class centers with high probability in sufficiently high dimension, giving a concrete width rule rather than choosing the number of binary heads heuristically.
Useful6/10
Difficulty4/10
Novelty5/10
Unverified
2026
Replace a dense channel-mixing matrix by a sparse complex generalised weighing matrix W with exactly w nonzero entries in every row and column, then use U=W divided by square root of w as a norm-preserving mixer. Restricting to k=2 gives a real matrix with entries in {+1,-1}; k=4 supports signed phase rotations. The exact isometry should preserve signal and gradient norms while reducing channel-mixing cost.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace a dense learned linear operator on continuous or image features by a truncated Hermite projection expansion whose coefficients are directly regularized in a Schatten-p norm. The layer becomes a structured low-rank operator, while the radial Hermite-Laguerre correspondence provides an analytically tractable parameterization and an exact spectral penalty.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Train a square orthogonal neural mixer while maximizing its entrywise fourth-power concentration. When optimization reaches a non-permutation stationary configuration, explicitly test rank-two row or column rotations and take a rotation with positive exact second variation, using the paper's constructive saddle-escape mechanism.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace a dense spatial parameter field in a neural field or convolutional adapter by a truncated squared-exponential KL expansion with analytic Gaussian-Hermite modes. The amplitude and correlation length remain trainable, but changing them only rescales coefficients and basis parameters instead of triggering a numerical eigensolve.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace a wide activation vector or spatial feature field by a small set of weighted coordinates that preserves the p-norm of every activation in a learned low-dimensional subspace. Unlike ordinary pruning, the selection objective is uniform over the whole coefficient sphere, so the compressed representation is designed to preserve unseen linear combinations and not merely the training examples.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Decompose tensor-valued hidden states into invariant symmetric, alternating, and higher Young-symmetry channels before applying learned maps. This removes redundant tensor coordinates and prevents a neural layer from mixing incompatible representation types, yielding smaller equivariant modules with a cleaner inductive bias.
Useful6/10
Difficulty4/10
Novelty5/10
✓✓ Beats tuned baseline
2026
Add an online low-rank reconstruction layer before a recommender or ranking MLP. It estimates a latent action-feature subspace from masked candidate vectors, freezes that subspace for an epoch, reconstructs each partially observed candidate in the latent coordinates, and feeds only those coordinates to the predictor. The method is most promising when the ambient candidate dimension d is large but the effective rank m is small and missingness is not too severe.
Useful6/10
Difficulty5/10
Novelty7/10
✗ Mechanism failed
2026
Replace a large dense layer whose input and output dimensions factor into multiple modes by a TT-matrix whose parameters are stored as a chain of small cores. Periodically apply TT-SVD rounding to remove weak singular directions and keep the representation within a prescribed approximation error. This transfers the paper's central computational principle—perform tensor-product contractions directly in compressed form—to neural network layers.
Useful6/10
Difficulty5/10
Novelty4/10
✗ Failed on benchmark
2026
Prune hidden units only after testing whether their nonlinear gate is task-visible and downstream-used. For ReLU, a unit is removable when its preactivation does not cross zero on the task patch or its outgoing weight column is zero; this is a more structural criterion than weight magnitude and can be applied during width search or post-training compression.
Useful6/10
Difficulty3/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Represent a large positive semidefinite neural operator as the sum of two Kronecker products and regularize an efficiently computed upper bound on its largest eigenvalues. The bound controls not only the spectral norm but every top-k eigenvalue sum, allowing a tunable penalty on concentrated or unstable directions without constructing the exponentially larger operator.
Useful6/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Replace nominal hidden-unit count by the number of distinct realized ReLU kink hyperplanes, then regularize or prune this effective count. Neurons whose normalized affine boundaries coincide can be exactly merged by summing their canonical coefficients, reducing memory without changing the represented function and aligning the model's complexity measure with the theorem.
Useful6/10
Difficulty5/10
Novelty6/10
✗ Mechanism failed
2026
Replace a dense weight matrix by a cross approximation built from selected rows and columns rather than by a conventional truncated SVD. Periodically refresh the selected indices using residual leverage scores, warm-starting from the previous factorization so that the compressed layer can track weight changes during fine-tuning.
Useful6/10
Difficulty5/10
Novelty5/10
✗ Mechanism failed
2026
Represent every mesh interface degree of freedom by one feature copy per incident cell, and apply local neural blocks directly to these cell tensors. Enforce inter-cell consistency with valence-weighted averaging only after selected layers or hierarchy transitions, avoiding repeated construction of a global sparse graph or assembled feature vector. This is suited to adaptive quadtrees, octrees, and finite-element neural operators.
Useful6/10
Difficulty5/10
Novelty7/10
✗ Mechanism failed
2026
Replace a dense channel or token-mixing matrix with a product of positive bidiagonal factors, so information propagates through a controlled sequence of local couplings rather than arbitrary signed interactions. Initialize the factors from the paper's barycentric-subdivision factorization, then learn positive diagonal and off-diagonal parameters; the resulting map is structured, parameter-efficient, and constrained to remain totally positive.
Useful6/10
Difficulty5/10
Novelty8/10
△ Mechanism confirmed, baseline not beaten
2026
Add a conservative correction after low-rank tensor compression so selected linear moments of an activation or learned state are exactly preserved. This can reduce tensor rank and memory without allowing compression error to accumulate in physically meaningful global quantities.
Useful6/10
Difficulty6/10
Novelty7/10
✗ Mechanism failed
2026
Replace dense grid tokens or global spectral features with coefficients of compactly supported kernels centered on a nested hierarchy of spatial points. Encode an input field into coarse-to-fine coefficients, apply a neural map to those coefficients, and decode the predicted coefficients at arbitrary query locations; the contribution from each level provides an explicit multiscale output decomposition.
Useful6/10
Difficulty6/10
Novelty6/10