Unverified
2026
Add a bank of temporal branches whose synapses apply heterogeneous delays before coincidence pooling. A branch selectively responds to an ordered pair or short sequence when the delay difference compensates the sequence interval, while low delay dispersion makes it primarily an order-invariant event detector. The branch threshold creates sparse, interpretable activations that can replace part of a recurrent or attention-based temporal module.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace an unconstrained attention or MoE routing matrix with a row-stochastic matrix that preserves a learned Gibbs-like distribution over groups. Anneal its temperature so that the router converges to a block-upper-triangular operator: mixing remains flexible within equal-energy groups, while cross-group traffic becomes directional rather than oscillating or collapsing. This creates a hierarchy-aware inductive bias and can reduce the number of active expert or attention connections.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace dense token attention with K asynchronous local nonlinear consensus updates on a sparse token graph. Each selected token is moved to the unique incident p-mean of its neighbors, producing a tunable message-passing operator that is averaging for p=2 and nonlinear for other p.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace ordinary feature-only cross-attention between two graph sets with a transport coupling optimized using pairwise structural costs. Use the paper's positive-semidefinite graph-kernel factorization to compute the quadratic GW interaction without materializing an O(N_0^2 N_1^2) tensor, and use the inexact-gradient convergence bound to control truncated or approximate updates. The resulting coupling can be used directly as attention weights or as a soft graph-matching matrix.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Construct a sparse directed attention mask over ordered tokens by treating token pairs as edges labeled by time or position. Apply a recursive three-hop spanner construction so that every dense temporal route has a sparse route using at most three times as many hops. Run attention only on the selected edges and use three sparse message-passing layers, or a small learned hop-composition block, to approximate dense causal attention.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace independent softmax routing or attention normalization with a differentiable approximate distribution over one-to-one assignments, using the Bethe permanent as the partition-function surrogate. Constrain the allowed token-to-expert or query-to-key support graph to have high girth, which gives an explicit bound on the approximation error and reduces short-cycle-induced correlations.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace dense attention weights with a compactly supported anisotropic bump derived from the obstacle solution, using one learnable ellipsoid per attention head or feature group. Tokens outside the learned ellipsoid receive exactly zero weight, while tokens inside receive smoothly decaying weights according to a fractional exponent. The learned positive-definite matrix represents orientation, scale, and correlations between feature dimensions.
Useful6/10
Difficulty6/10
Novelty5/10
Unverified
2026
Insert a sparse heavy-tailed nonlocal mixing operator into a residual sequence, graph, or spatial network so that information can traverse distant positions without stacking many local layers. Use the critical tail exponent s = 1/2, whose truncated first moment grows logarithmically and predicts an effective propagation distance proportional to depth times log depth rather than merely depth.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Use the paper's first-passage crossover as a controller for the communication or attention radius of a multi-agent policy. Instead of fixing a costly global attention range, estimate environmental depletion statistics online and set the radius near the predicted equality of individual and collective search times, expanding it only when the environment enters the collective-search regime. This turns a statistical-mechanical transition formula into an adaptive sparsification rule.
Useful6/10
Difficulty4/10
Novelty8/10
Unverified
2026
Parameterize a complex linear layer as a product of sparse triangular network factors whose positive modulus version is totally nonnegative. The layer can use phase cancellation for expressive transformations, while selected minors remain bounded by explicitly computable positive minors, giving a structured alternative to unconstrained dense complex weights.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace a sparse graph layer's separate edge transformations with one joint low-rank factorization of all transformations entering each target node. For target node i, concatenate the neighbor matrices horizontally, project all neighbor features into a shared low-dimensional receiving basis, and reconstruct one output; retain the self transformation exactly. This can reduce edge-parameter storage and message-passing FLOPs when the incoming block row has rapidly decaying singular values.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace unrestricted global attention or purely local convolution by a sparse distance-dependent interaction graph on a two-dimensional feature map. The edge probability or attention prior decays as \(r^{-(2+\sigma)}\), and \(\sigma\) becomes an explicit architectural control knob: small \(\sigma\) supplies mean-field-like global mixing, intermediate \(\sigma\) supplies long-range Wilson–Fisher behavior, and \(\sigma>2\) approaches a short-range model. The architecture should be evaluated not…
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Construct a sparse attention variant whose positional interaction kernel is piecewise polynomial rather than an arbitrary softmax score. Store key-value content in exact monomial moments within a spatial or learned-coordinate box hierarchy, then evaluate all queries from those moments without materializing the query-by-key matrix.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace repeated substrings in long sequences with nonterminal symbols from an acyclic straight-line grammar, then run the transformer on the compressed sequence. Unlike ordinary fixed tokenization, the compression objective explicitly minimizes the number of reusable binary productions, allowing repeated document-specific or corpus-level motifs to become single units. An expansion map lets the model recover token-level outputs for selected positions.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Represent hierarchical routing decisions by one weighted bipartite graph between parent experts or regions and fine cells or token groups. Select a regularized edge set once, then derive both coarse parent activation and fine-grained routing from it, preventing later refinement from invalidating earlier load balancing.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace flat expert logits with scores generated by Gaussian increments on a binary routing tree. A leaf receives the sum of increments on its root-to-leaf path, so sibling experts have correlated logits and the router can learn nested coarse-to-fine specialization; an inverse-temperature schedule controls the transition from exploratory diffuse routing to sparse routing.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace independent top-k token selection by a quality-weighted determinantal subset objective based on the Hardy–Szegő kernel. Tokens with high learned quality are preferred, but geometrically redundant tokens have a small determinant contribution, encouraging diverse sets of routed experts, retrieved items, or attended context tokens.
Useful6/10
Difficulty6/10
Novelty5/10
Unverified
2026
Parameterize a sequence attention graph by a finite graph motif on positions [r], then lift that motif to arbitrary sequence lengths using strictly increasing embeddings. Instead of learning an independent positional mask for every context length, the model learns a small collection of order-preserving edge types whose induced graph is valid at every length and can be extended without retraining.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace or augment a dense attention similarity matrix with a Min-cone matrix generated by a monotone scalar sequence. The resulting matrix is positive semidefinite by construction, has only O(n) learned scalar parameters, and can be multiplied by values in O(n d) time using cumulative sums rather than forming an n-by-n matrix.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Use the number of tokens participating in an attention operation to adapt the projected query/key dimension according to the sharp Johnson–Lindenstrauss bound. Short attention windows receive a dimension limited by n-1, while longer windows use a logarithmic dimension, avoiding a fixed large head dimension when the current attention set contains too few points to require it.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Add a distributed spectral positional encoding to a graph neural network, graph transformer, sparse-attention model, or MoE router by computing the dominant eigenvector of the current weighted adjacency matrix with a few warm-started power iterations. Unlike a Fiedler-vector feature, this encoding uses only local neighbor aggregation, is naturally nonnegative for nonnegative adjacency weights, and can be updated incrementally when the graph or edge weights change.
Useful6/10
Difficulty4/10
Novelty4/10
Unverified
2026
Replace a dense translation-invariant interaction matrix with a positive-definite Toeplitz kernel K_n(e^f) whose log-spectrum is parameterized by a small number of Fourier coefficients with 1/|k| decay. Use the paper's explicit quadratic term as a spectral-volume budget, allowing long-range structure while discouraging uncontrolled determinant growth and ill-conditioning. Subtracting this term from a log-determinant regularizer leaves a residual intended to capture higher-order deviations from…
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Make Laplacian latency attention hardware-friendly by quantizing the exponential decay and row normalization to powers of two. Exponentials and divisions can then be implemented with lookup tables and integer shifts, enabling a low-bit attention kernel for neuromorphic and edge accelerators.
Useful6/10
Difficulty4/10
Novelty7/10
Unverified
2026
Replace a freely learned finite impulse-response mixing kernel with a matrix polynomial whose roots are constrained to the unit circle. The resulting block-Toeplitz operator has an explicitly positive semidefinite spectral construction, while increasing the polynomial degree gives a systematic capacity knob for approximating matrix-valued frequency responses.
Useful6/10
Difficulty6/10
Novelty7/10