△ Mechanism confirmed, baseline not beaten
2026
Replace multiplicative query-key attention scores with an affinity based on the l1 distance between first-spike latency vectors. For each query token and key token, small latency differences produce large affinity and distant timings decay exponentially, yielding a locality-sensitive attention pattern naturally compatible with leaky spiking neurons.
Useful7/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Represent a graph, set, or attributed network as a measurable Z-valued kernel and train on lifted representatives while explicitly minimizing over node couplings. The quotient objective is invariant to relabeling by construction, while the lifted loss gives a dense correspondence signal that can stabilize graph attention and relational encoders.
Useful7/10
Difficulty5/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Freeze a randomly initialized single-layer transformer and use a constructed soft prompt to make its attention weights equal Gaussian-kernel weights over support examples. The resulting model performs Nadaraya-Watson regression in one forward pass, so task adaptation stores prompt tokens rather than modifying network weights. Prompt length becomes the number of kernel centers, while hidden dimension and prompt norm determine whether the required logits can be represented accurately.
Useful7/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace an n-by-n attention or token-mixing matrix with two nonnegative rank-r factors having row-simplex constraints and a shared latent column marginal. The induced matrix is exactly doubly stochastic at every accepted update, while applying it to values uses two thin matrix multiplications and never constructs the dense attention matrix.
Useful7/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace ordinary per-channel or per-token KV quantization with a structured orthogonal transform followed by blockwise 3-bit quantization. Use a normalized Walsh-Hadamard transform and small SO(4) rotations to spread outliers across coordinates, quantize the transformed vectors, and exploit orthogonality to rotate queries and attention outputs so unquantized attention remains mathematically equivalent.
Useful7/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace full agent-to-agent state transmission with coefficients in a learned dominant Koopman mode basis. Agents communicate only the leading spectral coordinates that explain slowly decaying collective behavior, while retaining a certificate based on the spectral gap and subdominant eigenvalue to decide whether the compressed representation is safe.
Useful7/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace globally unique node IDs with a small discrete alphabet labeling whose ordered local neighborhood words uniquely identify nodes. Feed these separating words to a GNN or graph transformer as positional features, preserving node distinguishability with bounded local receptive fields.
Useful7/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Add a per-feature or per-token state that accumulates recent stimulation and decays when stimulation is absent, then use a nonlinear decreasing gain to suppress repeatedly activated features. This creates short-term adaptation without changing the core transformer or recurrent weights: familiar inputs are processed with reduced gain, while novel inputs recover their full response.
Useful7/10
Difficulty4/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Use the relaxed QFT tensor-network topology as a trainable norm-preserving mixer inside a neural block, replacing a dense token-mixing matrix or an expensive global convolution. The network learns data-adapted global interactions while retaining structured O(N log^2 N) application and an exact cheap inverse, making it suitable for image tokens, long sequences, or reversible residual blocks.
Useful7/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace dense attention between tokens on opposite sides of a one-dimensional boundary or segment split with a dyadic low-rank approximation of a Cauchy/Hankel distance kernel. Each distance-scale block uses O(log(1/\varepsilon)) features, and the number of active scales grows only logarithmically with context length after discarding a narrow boundary layer. This is especially suitable for a relative-position attention branch or state-space-like long-range branch, rather than arbitrary…
Useful7/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Parameterize a multi-relational graph kernel as a finite stochastic block model and fit it by maximum entropy subject to differentiable motif-density constraints. Use the resulting block kernel as a graph-neural-network message-passing operator or structured prior for edge prediction, reducing an O(n^2 r) relation tensor to O(m^2 r+n) parameters for m latent blocks and r relations.
Useful7/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a portion of quadratic key-value attention or an external episodic table with a per-sample matrix fast memory updated by rank-one delta corrections. The memory directly learns a linear key-to-value map and can be carried across sequence segments, providing cheap online adaptation with constant state size per head.
Useful7/10
Difficulty5/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Replace or augment dot-product attention with a non-increasing radial kernel of pairwise representation distance. The bandwidth is normalized using an estimated local intrinsic dimension and local neighbor scale, creating an explicit locality-controlled attention operator.
Useful7/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Penalize short positive feedback cycles in an iterative neural module by suppressing products of absolute Jacobian blocks around the cycle. This targets the mechanism responsible for exponential temperature sensitivity rather than merely penalizing the total Jacobian norm, allowing strong feed-forward paths while controlling recurrent amplification.
Useful7/10
Difficulty7/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Train a matrix-valued neural layer under an exact or near-exact Stiefel constraint while using an l1 or row-group sparsity penalty. During early training, use manifold proximal-gradient steps to identify a stable nonzero support; once the support stops changing, switch to Newton-CG steps restricted to the smooth intersection of the Stiefel tangent space and the fixed-support subspace. This can reduce the number of optimizer iterations needed to obtain sparse, well-conditioned projections.
Useful7/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace dense query-key attention with an adaptive cross approximation constructed from selected query and key pivot tokens. At each rank, choose the pivot pair that removes large estimated residual energy, update the residual by a rank-1 cross correction, and stop when the residual estimate reaches a target tolerance. The resulting factorization computes approximate attention using a small number of landmark interactions while adapting to the actual token distribution.
Useful7/10
Difficulty6/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Replace ordinary low-precision multiply-add accumulation in selected neural-network reductions with a two-word floating-point accumulator updated by the paper's branch-free DW-FMA network. The high word retains the main sum and the low word stores the rounding residual, improving cancellation behavior without the control-flow divergence of conditional compensated summation.
Useful7/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace unconstrained token-mixing logits by a symmetric zero-row-sum response matrix generated from positive conductances on a small auxiliary electrical network. The resulting mixer has conservation and positivity structure, while circular minors have a prescribed sign pattern associated with positive grove measurements. This is especially suitable for graph neural networks and attention variants that need stable global diffusion rather than arbitrary dense affinities.
Useful7/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace selected dense neural-network operators by low-rank factors whose rank is selected by a randomized residual test at a user-specified tolerance. Construct candidate bases in large blocks for efficient matrix operations, then prune the block to the smallest rank that passes the residual criterion instead of treating the block size as the final rank.
Useful7/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Remove a latent relay or hub token from an attention or graph layer and replace its two-hop influence by direct effective edges between retained tokens. The correction is a normalized rank-one update, so it can preserve hub-mediated communication while reducing the number of stored and processed states.
Useful7/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
For a fixed structured graph, preprocess its adjacency matrix into the linear-time matrix-vector multiplication data structure guaranteed for classes of linear neighborhood complexity. Replace every dense aggregation Y=MX in a GNN by batched queries to this exact data structure, reducing a dense O(n^2d) aggregation to O(nd) after O(n^2) one-time preprocessing. This is especially useful for dense graphs from bounded-clique-width, bounded-expansion, minor-closed, twin-width, or related structured…
Useful7/10
Difficulty7/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a standard permutation-invariant object pool with a latent state on an unordered configuration together with a fiber vector transported along the observed object trajectories. The instantaneous state remains invariant to reordering, but loops and exchanges of objects act through learned monodromy matrices, allowing the network to represent path-dependent interactions without assigning arbitrary permanent object indices.
Useful6/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a large graph-token set by a smaller set of connected wedge regions generated through adaptive two-seed shortest-path partitions. Each pooled token is the mean of the node features in its region, while the binary partition tree and region sizes are retained for unpooling or skip connections. This provides a deterministic, graph-aware alternative to arbitrary token merging that can be inserted before graph-transformer message passing.
Useful6/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Represent a large positive semidefinite neural operator as the sum of two Kronecker products and regularize an efficiently computed upper bound on its largest eigenvalues. The bound controls not only the spectral norm but every top-k eigenvalue sum, allowing a tunable penalty on concentrated or unstable directions without constructing the exponentially larger operator.
Useful6/10
Difficulty5/10
Novelty7/10