Unverified
2026
Treat each neural-network block as a local strength system and measure how perturbations in its input channels affect multiple output observables, rather than using a single gradient norm. Use the estimated maximum directional gain to cap residual updates or assign a layerwise learning-rate multiplier, preventing weak high-gain layers from destabilizing training while allowing strong layers to move faster.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace expensive global spectral diagnostics of a cyclic or block-circulant neural layer by exact small Fourier-block calculations. Add a scale-normalized fourth-moment penalty, or directly cap the largest eigenvalue of each frequency block, to suppress frequency-specific amplification and reduce unstable training in long cyclic convolutions and structured attention.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Replace activation-magnitude-based adaptive computation halting with a criterion based on the actual recurrent update and a local stability margin. The loop halts when the state change is small relative to state scale for several consecutive steps, avoiding pathological decisions when LayerNorm-driven dynamics cause the activation norm to collapse.
Useful6/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Build a recurrent block as a fixed or learned ordering of local vertex foldings, mirroring the paper's identification of staircase solution maps with Coxeter elements of a folding group. Each folding changes one polygon coordinate by a rational cross-ratio completion while leaving all other coordinates unchanged. The resulting structured recurrence is reversible and can support constant-memory backpropagation by recomputing folds in reverse order.
Useful6/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a large graph-token set by a smaller set of connected wedge regions generated through adaptive two-seed shortest-path partitions. Each pooled token is the mean of the node features in its region, while the binary partition tree and region sizes are retained for unpooling or skip connections. This provides a deterministic, graph-aware alternative to arbitrary token merging that can be inserted before graph-transformer message passing.
Useful6/10
Difficulty5/10
Novelty6/10
✗ Failed on benchmark
2026
Construct a sparse token-interaction graph from attention affinities and recursively split it by removing the closed neighborhoods of a small set of separator tokens. Separator tokens retain global communication, while the resulting anti-adjacent components perform local attention independently, reducing quadratic attention and KV-cache costs. The induced Erdos-Pósa theorem supplies a structural diagnostic: graphs with few anti-adjacent long-cycle or theta packings should admit small…
Useful6/10
Difficulty7/10
Novelty8/10
✗ Mechanism failed
2026
Replace a dense weight matrix by a cross approximation built from selected rows and columns rather than by a conventional truncated SVD. Periodically refresh the selected indices using residual leverage scores, warm-starting from the previous factorization so that the compressed layer can track weight changes during fine-tuning.
Useful6/10
Difficulty5/10
Novelty5/10
✗ Mechanism failed
2026
Turn attention weights into a Boolean support scenario and prune edges using local-surjectivity constraints rather than independently thresholding each row. Preserve at least one compatible continuation for every local window, then favor a strongly connected support graph so pruning removes redundant mixtures while maintaining a globally coherent sparse attention pattern.
Useful6/10
Difficulty5/10
Novelty7/10
✓ Mechanism works
2026
Replace dense attention between grid-arranged tokens by a deterministic block-sparse pattern generated from modular permutations. In each block, connect row token i to column token p(i)=2i modulo B; because i, i-p(i), and i+p(i) are injective modulo B when gcd(B,6)=1, the pattern avoids repeated horizontal, vertical, and diagonal projections. Use shifted permutations across heads to increase receptive-field coverage while retaining structured sparsity.
Useful6/10
Difficulty4/10
Novelty6/10
✗ Mechanism failed
2026
Bootstrap the optimizer curvature scale from a deliberately nondegenerate pair of gradient queries, then perform steepest descent in lp geometry with a local secant backtracking rule. The method does not require a supplied learning rate, smoothness constant L, initial distance R, or optimum value f*, and it automatically uses the dual norm associated with p.
Useful6/10
Difficulty5/10
Novelty6/10
Audited (legacy)
2026
Use the attention probability distribution over an ordered context to choose contiguous token groups whose pooled attention masses have entropy as close as possible to a prescribed upper budget R. Replace the corresponding key/value vectors by one weighted representative per group, preserving token order and reducing the KV-cache length from n to m. Unlike unconstrained token merging, the entropy constraint gives a direct control knob over how concentrated or diffuse the retained attention…
Useful6/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace dense token-to-token attention by a learned binary relation generated from a small number of hierarchical predicates, while rejecting masks that contain a fixed K_{t,t} biclique. The paper's incidence bound predicts near-linear active edges for these structured relations, giving sparse attention with a measurable worst-case complexity target rather than relying only on average sparsity.
Useful6/10
Difficulty6/10
Novelty6/10
✗ Mechanism failed
2026
Replace ordinary masked mean pooling with a Fourier-compressed quadrature operator for arbitrary two-dimensional or three-dimensional domains. The geometry is preprocessed once into reusable grid weights, allowing every channel and every training example using the same domain to be pooled without boundary-area bias.
Useful6/10
Difficulty3/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Use the weighted quadrature identity as a training or inference constraint for a compressed activation path: retain only a minimal set of binary evaluations and compute normalization or residual-energy statistics exactly on the modeled Rademacher component. This provides a zero-variance alternative to random activation subsampling for the represented subspace.
Useful6/10
Difficulty4/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Construct a sparse attention support by solving multiple small perturbed assignment problems between query and key embeddings and taking the union of the selected optimal matchings. Use the resulting spanning tree as the only set of cross-token edges, with edge biases determined by empirical assignment frequency.
Useful6/10
Difficulty7/10
Novelty7/10
Unverified
2026
Replace a dense or ad hoc sparse attention pattern with a circulant mask generated by the paper's carry word c(m,k). Every query attends to exactly m of k relative positions, and the selected positions are prefix-balanced, avoiding the large gaps and collisions produced by random sparsification. Use several Farey-related slopes across heads or layers to combine local, medium-range, and long-range coverage.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Use the adjacency matrix of a vertex-transitive strongly regular graph as a fixed sparse attention or token-mixing mask. Every vertex has the same degree, and every pair of vertices has exactly one of two common-neighbor counts, giving predictable two-hop coverage and avoiding the degree and connectivity irregularities of random sparsification.
Useful5/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace a dense token- or channel-mixing matrix with a product of local braid generators acting on adjacent coordinates. Each generator is an exactly invertible 2-by-2 transformation, while the braid and far-commutativity identities give multiple equivalent factorizations of the same global operator. This creates a sparse, reversible mixer with O(kn) cost for a braid word of length k, rather than O(n^2) cost for a dense matrix.
Useful5/10
Difficulty4/10
Novelty7/10
Unverified
2026
Build a binary hierarchy over tokens by recursively splitting each active block with a beta-splitting rule, then perform dense attention only inside small leaf blocks and communicate between leaves through learned summaries at internal nodes. The beta parameter controls how balanced the partition is, while the paper's maximum-depth asymptotic supplies a principled depth budget and a way to detect pathological trees.
Useful5/10
Difficulty6/10
Novelty5/10
Unverified
2026
Use the paper's eventual path-length bounds to constrain an order-invariant routing graph to a constant-hop communication budget. A learned sparse attention or graph-neural-network layer can explicitly route information through at most three admissible hops, while a more conservative auxiliary route permits at most five minimal-path hops, preventing increasingly long and unstable dependency chains as sequence length grows.
Useful5/10
Difficulty7/10
Novelty8/10
Unverified
2026
Replace dense token-to-token attention on a 2D token grid with local attention plus sparse horizontal and vertical communication axes. Tokens at intersections of selected axes receive extra cross-axis attention edges, creating a reinforced sparse graph that can transmit information across large blocks while using far fewer edges than dense attention. The mask should use light-tailed, approximately geometric spacing in both directions rather than heavy-tailed spacing in one direction.
Useful5/10
Difficulty5/10
Novelty6/10
Unverified
2026
Constrain a sparse attention graph to split into a k-degenerate backbone and a residual graph with maximum degree at most k-1. Orient the backbone according to a degeneracy order so that each token receives or emits at most k structured interactions in the relevant direction, while the residual edges form a bounded-degree correction layer. This replaces arbitrary sparse attention with a topology that is easier to schedule and whose worst-case edge and local-degree costs are explicit.
Useful5/10
Difficulty6/10
Novelty8/10
Unverified
2026
Convert attention scores into binary incidence set systems at one or several score thresholds, then group queries that induce the same sampled-key pattern. Compute the expensive key-value aggregation once per pattern and reuse it for all queries in the group. The method is exact for the thresholded routing component and approximates dense attention when queries have a small number of stable high-weight neighborhoods.
Useful5/10
Difficulty6/10
Novelty7/10
Unverified
2026
Construct a sparse token-mixing architecture by interleaving learned per-token transformations with fixed perfect-shuffle and cyclic-pile permutations. For n not a power of k, the generated permutation group is 2-transitive, so sufficiently rich sequences of generator words can expose every ordered token pair without constructing a dense N by N attention matrix.
Useful5/10
Difficulty4/10
Novelty6/10