△ Mechanism confirmed, baseline not beaten
2026
Replace a quadratic pairwise attention or graph aggregation kernel with a compact, translation-invariant indefinite kernel approximated by signed random Fourier features. The feature map preserves the kernel's negative spectral mass through a diagonal sign matrix, so the resulting linear-time aggregation can represent similarities that ordinary positive-definite random features cannot.
Useful7/10
Difficulty5/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Replace dense graph self-attention with two parallel branches: exact softmax attention only over graph neighbors and a global linear-attention branch that summarizes all nodes through feature-space statistics. A learned node-wise gate interpolates between the branches, allowing locally structured nodes to use sparse attention while retaining a global-information path.
Useful7/10
Difficulty4/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Construct a cheap graph or token-mixing operator by partitioning nodes into k blocks using the bottom nonconstant eigenvectors of P squared, then replacing dense pairwise mixing with conditional averaging inside each block followed by one baseline propagation step. Unlike ordinary spectral clustering, the bottom modes target partitions where block labels are rapidly destroyed by P, producing an aggressively mixing representation layer rather than a community-preserving pooling layer. The…
Useful7/10
Difficulty6/10
Novelty7/10
✗ Mechanism failed
2026
Replace Euclidean momentum for selected neural parameters with a mirror or Bregman update, while using the paper's accelerated Newton direction for the objective step. Entropy geometry is especially suitable for softmax MoE routers, while Euclidean or log-barrier geometries can be used for unconstrained or positive parameters.
Useful7/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Replace an explicit attention step by an implicit Euler step that solves a proximal subproblem involving the Hopfield energy. The new state is evaluated inside the softmax self-consistently, which makes the method less sensitive to large step sizes and can prevent explicit attention from overshooting or tunneling between attraction basins.
Useful7/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace flat all-pairs attention with attention neighborhoods induced by a compatible tree over tokens, patches, nodes, or retrieved items. Retain exact or approximate attention inside nearby tree subtrees and add a path-monotonicity regularizer so semantically distant endpoints are not more similar than intervening tree neighbors.
Useful7/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace full PSD self-attention with a pivoted Cholesky/Nyström approximation whose landmarks are sampled from the unexplained diagonal mass. Tokens with large residual self-similarity are more likely to become landmarks, so the rank budget is spent on difficult regions rather than uniformly selected tokens.
Useful7/10
Difficulty5/10
Novelty5/10
✗ Mechanism failed
2026
Choose the neural operator's input-history length from the measured correlation time of the unresolved closure signal produced by coarse-graining. This avoids under-memory, which causes systematic closure error, and over-memory, which increases attention cost and can destabilize training. The same diagnostic can drive adaptive memory truncation across physical regimes.
Useful7/10
Difficulty4/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Use a learned scalar ordering function to turn a symmetric local Gaussian graph kernel into a directed, row-stochastic message-passing operator. The asymmetric tilt lets neighboring nodes communicate preferentially along an inferred dynamical direction, while the Gaussian factor retains locality and diffusion-like smoothing.
Useful7/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Train a fixed-rank neural weight update Y=USV^T with a projector-splitting Runge–Kutta step instead of independently applying Adam or gradient descent to U, S, and V. The update evolves the full low-rank matrix using the neural gradient but performs QR-based factor updates, avoiding S^{-1} and remaining stable when adapter singular values collapse or cross zero. Use a common-base midpoint construction so every internal stage starts from the same U,V basis and remains rank r.
Useful7/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Add a weighted reflection symmetry to an attention or graph-propagation matrix instead of requiring ordinary permutation equivariance. For paired positions or graph nodes related by an involution, penalize the failure of the propagation operator to commute with the weighted reflection; this makes all geometric multi-step propagations symmetry-compatible. The method is suitable for data with mirror, reversal, paired-agent, or left/right structure where the two sides have unequal importance…
Useful7/10
Difficulty4/10
Novelty6/10
✗ Mechanism failed
2026
Prune redundant attention heads using separate similarity scores for sink behavior and content routing. Two heads are considered safely redundant only when their normalized content compositions are close in Aitchison distance and their sink-mass trajectories are also close, avoiding pruning decisions dominated by a shared sink token.
Useful7/10
Difficulty4/10
Novelty7/10
✗ Mechanism failed
2026
Distill a teacher's attention into a student by matching sink mass and the normalized content distribution as separate targets rather than applying one KL divergence to the entire attention row. Use the Aitchison distance on the content composition, which compares relative token allocation and prevents a large common sink probability from overwhelming differences between content tokens.
Useful7/10
Difficulty3/10
Novelty7/10
✗ Failed on benchmark
2026
Insert a linear Johnson–Lindenstrauss bottleneck around a set of jointly processed representations, choosing its width from the sharp finite-set dimension bound rather than from the model's nominal hidden size. The projection should preserve pairwise distances between tokens, patches, or retrieved items, allowing a downstream attention or MLP block to operate at lower width while retaining the geometry relevant to similarity computations.
Useful7/10
Difficulty5/10
Novelty5/10
✗ Failed on benchmark
2026
Parameterize an entropic OT cost only in directions that can change the transport plan, removing row-plus-column potential directions that are invisible because of OT gauge invariance. Whiten the remaining feature coordinates using their empirical covariance, producing an OT layer whose identifiable parameters have substantially more uniform sensitivity.
Useful7/10
Difficulty5/10
Novelty7/10
✗ Mechanism failed
2026
Replace dense cross-attention weights with a balanced transport plan whose nonzero query-key edges are maintained by a multiscale active-set procedure. Solve the coarse token-group problem first, lift its support to the fine token grid, add only edges indicated by local cost or marginal residuals, and warm-start the fine problem from the lifted plan. This should provide a principled sparse attention pattern rather than fixing a global top-k pattern before seeing the transport solution.
Useful7/10
Difficulty7/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace multiplicative query-key attention scores with an affinity based on the l1 distance between first-spike latency vectors. For each query token and key token, small latency differences produce large affinity and distant timings decay exponentially, yielding a locality-sensitive attention pattern naturally compatible with leaky spiking neurons.
Useful7/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Represent a graph, set, or attributed network as a measurable Z-valued kernel and train on lifted representatives while explicitly minimizing over node couplings. The quotient objective is invariant to relabeling by construction, while the lifted loss gives a dense correspondence signal that can stabilize graph attention and relational encoders.
Useful7/10
Difficulty5/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Replace ordinary per-channel or per-token KV quantization with a structured orthogonal transform followed by blockwise 3-bit quantization. Use a normalized Walsh-Hadamard transform and small SO(4) rotations to spread outliers across coordinates, quantize the transformed vectors, and exploit orthogonality to rotate queries and attention outputs so unquantized attention remains mathematically equivalent.
Useful7/10
Difficulty6/10
Novelty6/10
✗ Failed on benchmark
2026
Use predicted covariance reduction as a differentiable gate for selecting tokens, views, sensors, or retrieved demonstrations. The gate favors inputs with high expected information gain while accounting for acquisition cost, turning attention and data collection into active observability optimization.
Useful7/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace full agent-to-agent state transmission with coefficients in a learned dominant Koopman mode basis. Agents communicate only the leading spectral coordinates that explain slowly decaying collective behavior, while retaining a certificate based on the spectral gap and subdominant eigenvalue to decide whether the compressed representation is safe.
Useful7/10
Difficulty6/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Build a kernel aggregation layer whose output is a tangent vector field on the unit sphere and whose surface divergence is identically zero by construction. For each source point, use a matrix kernel obtained by applying a surface-rotated gradient in the query variable to a scalar zonal kernel; this is a differential-form version of the paper's matrix-valued construction. The layer can replace attention or message passing when the target dynamics are incompressible, such as spherical fluid…
Useful7/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace globally unique node IDs with a small discrete alphabet labeling whose ordered local neighborhood words uniquely identify nodes. Feed these separating words to a GNN or graph transformer as positional features, preserving node distinguishability with bounded local receptive fields.
Useful7/10
Difficulty5/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Represent each input graph by a short ordered sequence of topological tokens rather than a persistence diagram. For each overlapping interval on a shared filtration grid, compute the number of connected components and independent one-dimensional cycles, concatenate these counts with inexpensive graph summary statistics, and process the resulting sequence with a small Transformer. The representation should retain multiscale structural information while avoiding pairwise persistence-diagram…
Useful7/10
Difficulty6/10
Novelty6/10