Unverified
2026
Represent a neural-network weight tensor by rank-one terms whose mode factors are selected from shared orthonormal bases, and impose the same basis alignment across tensor flattenings. During or after training, retain the largest coefficients to obtain a structured truncation analogous to truncated SVD. This should produce better-conditioned tensorized layers than unconstrained CP factors while preserving a directly controllable accuracy/compute tradeoff.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Replace a learned sequence-mixing matrix with a structured lower-triangular Sprugnoli operator whose square is exactly the identity. Applying the same operator in reverse reconstructs activations exactly, so it can be used as a reversible Transformer mixer or reversible channel permutation while retaining nontrivial long-range mixing.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Build a neural feature-mixing block from a sparse exchange matrix and a fixed sequence of cluster-style mutations. Transpose-related channels share parameters through orbit tying, while positive multiplicative-additive updates are evaluated with log-sum-exp. The block is a concrete alternative to a dense MLP projection and can be tested at equal parameter count and compute.
Useful6/10
Difficulty5/10
Novelty8/10
Unverified
2026
Construct a symmetric token affinity matrix and approximate only its spectrally outlying token-mixing modes using a small set of sampled landmark columns. The resulting low-rank operator replaces an \(O(n^2)\) dense mixer by two skinny matrix multiplications, while the paper's residual guarantee predicts that large-magnitude global interaction modes are preserved.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Replace a learned nonnegative banded transition matrix by an ordered product of learnable stochastic bidiagonal factors. Each factor performs one local left or right transport step, so a product of p lower and q upper factors creates an effective bandwidth of p+q while retaining nonnegative entries, row sums equal to one, and a highly structured propagation kernel.
Useful6/10
Difficulty3/10
Novelty7/10
Unverified
2026
Replace unconstrained low-rank compression of a neural state with an augmented basis that always contains vectors representing known conserved quantities or diagnostically important linear statistics. After each learned transition, project the state back onto the affine constraint set with an exact minimum-norm correction, preventing rank truncation and model error from accumulating in those statistics.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace dense spatial attention on a 2D feature grid by attention from every non-landmark token to a small periodic set of landmark tokens. Choose landmarks using the paper's residue construction so that every non-landmark token has at least one landmark exactly k Manhattan steps away. Use this guaranteed edge as a sparse global-information route, combined with a cheap local convolution or residual path for accuracy.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Construct one reference Lanczos basis for a symmetric propagation operator H, then derive several seed-specific spectral responses for Q_a(H)x_0 through finite-band polynomial connectors. With degree-r seeds, each transformed basis vector uses at most 2r+1 neighboring reference basis vectors, avoiding a separate Lanczos factorization for every seed.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace an SVD-based pseudoinverse of a learned rectangular matrix with a low-degree polynomial initialization followed by a few Newton–Schulz iterations. The polynomial approximates the inverse Gram operator, while a cheap residual test accepts it only when the iteration is contractive and otherwise selects a conservative transpose-scaled initialization.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace random or degree-based graph positional anchors with a greedy set chosen to maximize the entropy of the graph's Weisfeiler-Lehman equivalence partition. Give each selected anchor a unique trainable embedding, then run several WL-inspired refinement rounds to produce positional cell features. The resulting positional signal should distinguish nodes that ordinary message passing treats identically while using only a small anchor budget.
Useful6/10
Difficulty4/10
Novelty5/10
Unverified
2026
Represent a multiparameter feature stream as a direct sum of interval modules over the nonempty-subset lattice of several corruption, scale, or context variables. Each learned channel is active only on a connected order-convex region of the lattice, replacing a dense bank of unrelated per-condition adapters with reusable feature channels having explicit parameter-space lifetimes.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace a dense Gaussian or learned projection from dimension N to m with a normalized partial circulant projection generated by a single Gaussian vector. For K-sparse hidden states, the restricted-isometry guarantee predicts approximate norm preservation while reducing stored projection parameters from O(mN) to O(N). The projection can be evaluated with an FFT and should be combined with explicit top-k gating so that the sparse-input assumption is enforced.
Useful6/10
Difficulty4/10
Novelty5/10
Unverified
2026
Replace direct source-to-target attention connectivity by two sparse incidence relations through a set of latent witness tokens. A source token attends only to a small set of witnesses, and each witness attends only to a small set of target tokens; the composed relation is trained to contain exactly one witness for desired pairs and no witnesses for undesired pairs. This produces a controllable sparse attention pattern whose errors can be measured entrywise against a dense teacher or known mask.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace a collection of overlapping sliding-window features with approximately orthogonal incremental features: the length-m feature contains information not predictable from shorter consecutive windows. Use the paper's transition-operator Toeplitz precision matrix to decorrelate the resulting sequence of window features before attention, suppressing duplicated local evidence and improving conditioning.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Construct a graph on tokens using a cheap similarity or k-nearest-neighbor rule, partition it into small-radius pieces, and use one pooled state per piece for cross-piece communication. The paper's bound guarantees that the total number of vertices exposed by all one-hop piece neighborhoods is at most 4n, so the routing graph can have linear rather than quadratic communication budget. The approximation is controlled by increasing the number of within-piece basis summaries rather than making…
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Attach a small dynamical observer to a neural ODE, RNN, or state-space model and make it estimate only a task-relevant functional of the hidden state, such as logits, value features, or control-relevant projections. Use an incremental quadratic constraint and a bounded-real penalty to make the observer robust to hidden-state nonlinearities and input disturbances, instead of reconstructing the full latent state.
Useful6/10
Difficulty6/10
Novelty5/10
Unverified
2026
Replace a dense order-p tensorized neural layer with a sparse random hypergraph tensor whose support probability is alpha = c/N^{p-1}. Assign independent zero-mean marks to active hyperedges, so every tensor moment scales as N^{-(p-1)} while the total variance entering each output remains controlled. The resulting locally tree-like computation graph should reduce parameters and multiply-adds without immediately destroying signal propagation.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Construct sparse attention masks as regular or almost-regular hypergraphs: every query selects exactly w keys, while every key receives nearly equal incidence within each score or locality class. This preserves a fixed sparsity budget while removing systematic positional and head-level load imbalance that can cause hot keys, uneven memory traffic, and unstable gradients.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace a recurrent sequence classifier's unconstrained hidden-state alarm head with an online truncated-signature state and a first-hitting-time linear detector. The module summarizes local order information and cross-channel interactions while preserving exact compositional updates, making it suitable for long streaming sequences and early-exit decisions.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Turn the paper's graphon refinement operation into an adaptive mixture-of-experts or relational-attention router whose number of blocks is learned rather than fixed. Split a block when its members produce incompatible constraint gradients, and merge blocks when their relation rows are statistically indistinguishable, yielding a compact architecture that grows only where the data require extra structure.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace a large stable linear state-space or recurrent layer by a lower-order balanced realization computed from frequency-targeted controllability and observability Gramians. Use generalized low-rank ADI with imaginary-axis shifts concentrated at frequencies that dominate the training data, then retain states associated with the largest approximate Hankel singular values. This should reduce recurrent inference cost while preserving the layer's input-output response in the selected frequency…
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Use the paper's correspondence between KAN splines and finite-element or isogeometric shape functions to build coordinate-separable tensor-product trial layers. Replace additive coordinate aggregation with a multiplicative contraction of one-dimensional spline expansions, yielding an explicit tensor-product basis without storing a dense multidimensional grid.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace fixed sequence-to-sequence attention with a dynamically maintained tree of connected token groups. Groups can be fused to reduce the number of attention units or split when their representation is heterogeneous, while hypergraph connectivity and nestedness ensure that every intermediate hierarchy remains valid.
Useful6/10
Difficulty6/10
Novelty5/10
Unverified
2026
Replace a dense tensor-product positional encoding or first MLP layer with a hierarchical sparse-grid B-spline feature map. The network evaluates only localized basis functions indexed by multi-levels with bounded total level, reducing feature count while retaining high-order approximation for functions with mixed derivative regularity. The basis can initially be fixed and later fine-tuned jointly with the downstream network.
Useful6/10
Difficulty5/10
Novelty6/10