Unverified
2026
Replace a dense attention pattern by the exact intersection of a fixed or cheaply computed base graph H and a learned shared-label relation. Two tokens can exchange information only when they are adjacent in H and share at least one of d labels, producing a controllable structured sparsity pattern. The label count d becomes an explicit capacity and compute knob: increasing d enlarges the relation vocabulary without requiring a dense pairwise mask.
Useful5/10
Difficulty6/10
Novelty5/10
Unverified
2026
Replace part of an attention matrix with a mixture of fuzzy permutation matrices induced by short permutations. Each basis element represents an order-preserving k-token matching smeared over all embeddings into the sequence, while a balancing constraint makes the aggregate attention receive uniform global coverage. Retain a standard low-rank or local-attention residual so the structured branch does not prevent arbitrary content-dependent interactions.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Regularize a hard MoE router so that assignments remain block-jumbled: every group of token positions sends approximately the expected number of tokens to every group of experts or capacity slots. The condition detects localized routing collapse that ordinary global load balancing can miss, while requiring only a small block-count matrix rather than expensive pairwise or pattern statistics.
Useful5/10
Difficulty4/10
Novelty5/10
Unverified
2026
Replace or augment an exponential state-space memory branch with a causal convolution whose lag-j weight is exp(-lambda j) ell(j)/j. The 1/j boundary provides broad logarithmic memory, while lambda supplies an explicit finite memory scale and prevents uncontrolled accumulation from an untempered long-memory kernel.
Useful5/10
Difficulty5/10
Novelty5/10
Unverified
2026
Use the model's non-monotonicity result to make graph connectivity a learned resource rather than assuming that every extra edge helps. An edge router assigns transmission scores but also charges a source-side collision cost for exposing an infected node to many susceptible neighbors. The resulting router can prune edges that increase competition and reduce useful reachability.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace independent softmax expert choices with a collision-free Markov router whose particles occupy expert positions on a one-dimensional or circular index lattice. A particle can move only to an empty neighboring expert, and the move rate contains a product of sine ratios that globally repels nearby assignments; this should reduce expert collapse and produce more evenly spread routing without requiring a separate pairwise diversity loss.
Useful5/10
Difficulty7/10
Novelty8/10
Unverified
2026
Construct a nested sequence of representations in which each coarser representation is obtained from the previous one by a 1-Lipschitz projection. Train prediction heads at multiple scales so coarse predictions remain stable and approximately recoverable from the finer representation, enabling early exit, token pooling, and controlled multiresolution inference.
Useful5/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace part of dense token mixing with a small bank of structured permutations acting on a hierarchical token tree. Diagonal inheritance shares the same local permutation across all descendant copies, while selector words activate one connector type at a chosen level and remain inactive on the next type, providing controllable multiscale receptive fields without constructing a dense attention matrix.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Partition a sequence, image, or graph into regions processed by a cheap local operator and a more expressive nonlocal operator, then couple their boundary activations with a shared continuity equation and a conservative interface-flux equation. The interface correction prevents the local and global branches from creating discontinuities or duplicated information, allowing nonlocal computation to be restricted to selected regions while preserving global consistency.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace one dense attention layer with a sparse hierarchical attention module whose states are clusters of tokens and whose transitions merge two clusters or reverse a previous merge. Enforce the flag-space cancellation law on pairs of alternative two-step merge paths, so redundant hierarchical routes destructively interfere instead of producing duplicated features. Normalize merge-then-unmerge loops using the product of the sizes of the merged clusters, preventing large clusters from…
Useful5/10
Difficulty7/10
Novelty8/10
Unverified
2026
Represent sequence positions as vertices on a circle and use a maximal family of mutually non-crossing d-arcs to define the allowed attention interactions. Rotate the family by d positions, or use several phase-shifted families across successive layers, so each layer has only O(N) edges but repeated layers propagate information over long distances. The geometric compatibility rule replaces arbitrary local-window or hand-designed sparse masks with a structured family whose maximality gives broad…
Useful5/10
Difficulty4/10
Novelty7/10
Unverified
2026
Replace one-step greedy landmark selection in Nyström attention or kernel compression with a restricted pairwise-lookahead rule. The lookahead is motivated by the paper's explicit obstruction: a signed triangle can make individual column gains exhibit increasing rather than diminishing returns, so the best next column need not belong to the best pair.
Useful5/10
Difficulty5/10
Novelty4/10
Unverified
2026
Use the paper's degree-sensitive crown inequality to penalize or constrain router assignments that create medium- or high-degree tokens or experts. The resulting router favors a controlled population of low-degree, medium-degree, and high-degree nodes rather than allowing a few hubs to absorb most interactions, which can stabilize sparse attention or mixture-of-experts load balancing.
Useful5/10
Difficulty5/10
Novelty5/10
Unverified
2026
Use the isolated positive spectral mode created by a finite branch defect on an otherwise long cycle as a graph positional feature. The feature should concentrate around structurally unusual vertices while remaining insensitive to the total cycle length, providing a principled alternative to raw Laplacian eigenvectors for cycle-with-branch graphs.
Useful5/10
Difficulty5/10
Novelty8/10
Unverified
2026
Replace a dense or irregular binary interaction matrix in a value-aggregation layer with a signed sum of blocky masks. Each blocky mask groups a set of query rows with a disjoint set of key columns, allowing all queries in a group to reuse one summed value vector. This is most suitable for linear attention, graph message passing, or any layer where the interaction matrix is applied directly to values rather than passed through a row-wise softmax.
Useful5/10
Difficulty7/10
Novelty7/10
Unverified
2026
Learn or select sparse cyclic convolution or relative-attention offsets whose pairwise differences collide less often modulo the sequence length. The paper's Fourier fourth-power identity turns this combinatorial objective into an FFT-computable differentiable loss, enabling fixed-K sparse patterns with lower aliasing and interference than random offsets.
Useful5/10
Difficulty4/10
Novelty7/10
Unverified
2026
Use eigenvector delocalization as a mask-quality criterion rather than selecting a random sparse graph blindly. Penalize masks whose normalized adjacency has concentrated leading eigenvectors or disconnected or weakly connected components, while preserving the power-law distance prior. This creates a sparse routing graph that is less likely to trap information in local regions.
Useful5/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace or augment relative-position attention with a positive fractional-integration mixing kernel whose radial behavior has separate inner and outer power laws. Tokens close to one another interact through the usual fractional singularity, while tokens near different radial scales receive a ground-state correction that can improve multiscale information transport without introducing a dense learned positional table.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace independent top-k expert or attention-edge selection with a gate vector constrained by hierarchical path budgets modeled on the paper's extended-Dyck-path polytope. Ordinary interactions receive continuous nonnegative capacities, while a designated class of cross-group interactions receives binary or clipped-to-one gates, producing structured sparsity and preventing many correlated paths from consuming the same routing budget.
Useful5/10
Difficulty6/10
Novelty7/10
Unverified
2026
Build a neural architecture whose receptive field or attention span is increased according to an estimated disorder-to-order crossover scale. Local branches process windows below the crossover as if they were stochastic, while a global branch is activated only when the context exceeds the predicted scale needed to expose deterministic recurrence. This targets sequences or images containing long-range quasiperiodic, hierarchical, or algorithmically generated structure that is statistically…
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace softmax attention or dense MoE routing with a normalized Rayleigh–Jeans distribution over tokens or experts. If an item's energy is close to the chemical potential, its probability becomes disproportionately large, creating controllable low-energy condensation instead of the exponentially smooth allocation produced by softmax.
Useful5/10
Difficulty6/10
Novelty7/10
Unverified
2026
Represent many related sparse graph or attention patterns inside one fixed host connectivity pattern and activate each target instance with binary directional masks. The learned edge transformation and sparse-kernel layout are shared across instances, while the mask selects the target graph, enabling one compiled operator to process heterogeneous structures.
Useful5/10
Difficulty5/10
Novelty5/10
Unverified
2026
Construct a candidate feature for every edge pair or structured token pair, then retain a numerically independent subset under a feature-Jacobian matroid. The neural layer computes only the selected interactions, preserving directions that add new information rather than pruning solely by magnitude or attention score.
Useful5/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace a generic three-input concatenation MLP with a permutation-symmetric mixer built from the four signed combinations x+y-z, x-y+z, -x+y+z, and -x-y-z. Apply a shared truncated exponential to these combinations and aggregate symmetric pairwise products, producing controlled quadratic and higher-order interactions without materializing a full trilinear tensor.
Useful5/10
Difficulty4/10
Novelty7/10