✓✓ Beats tuned baseline
2026
Replace dense self-attention by a sparse attention graph whose neighborhoods satisfy the paper's size-dependent expansion condition. This preserves a logarithmically controlled route for every token subset to communicate with the rest of the sequence, reducing quadratic attention cost without allowing disconnected or poorly mixed token groups.
Useful8/10
Difficulty5/10
Novelty5/10
✓✓ Beats tuned baseline
2026
Give every possible pairwise relation its own recurrent latent state, stored in a dictionary keyed by stable node identities, instead of discarding the state whenever the edge is absent from the current graph. At each timestep, active edges retrieve their old state, update it with the current pair features and neighborhood messages, and write it back after message passing. This should improve predictions in dynamic graphs with intermittent interactions and reduce the burden on node states to…
Useful7/10
Difficulty5/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Replace dense graph self-attention with two parallel branches: exact softmax attention only over graph neighbors and a global linear-attention branch that summarizes all nodes through feature-space statistics. A learned node-wise gate interpolates between the branches, allowing locally structured nodes to use sparse attention while retaining a global-information path.
Useful7/10
Difficulty4/10
Novelty5/10
✓✓ Beats tuned baseline
2026
Replace ordinary Wasserstein or arithmetic pooling of distribution-valued features with a barycenter whose individual quantile displacements are Huberized. Small changes between input distributions remain averaged quadratically, while a corrupted token, expert, graph neighborhood, or augmentation cannot move the pooled distribution arbitrarily far. The module is especially cheap for one-dimensional distributions represented by fixed quantile vectors.
Useful7/10
Difficulty4/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Build a kernel aggregation layer whose output is a tangent vector field on the unit sphere and whose surface divergence is identically zero by construction. For each source point, use a matrix kernel obtained by applying a surface-rotated gradient in the query variable to a scalar zonal kernel; this is a differential-form version of the paper's matrix-valued construction. The layer can replace attention or message passing when the target dynamics are incompressible, such as spherical fluid…
Useful7/10
Difficulty5/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Replace dense token-to-token attention by a multiscale spiderweb communication pattern. Tokens first aggregate upward through a dyadic hierarchy, communicate horizontally only with a small number of cells at the appropriate height, and then receive information broadcast downward. Hyperbolic distance supplies a principled rule for choosing the height at which two tokens interact: nearby tokens interact at fine scales, while far-apart tokens interact through coarse representatives.
Useful7/10
Difficulty5/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Represent each input graph by a short ordered sequence of topological tokens rather than a persistence diagram. For each overlapping interval on a shared filtration grid, compute the number of connected components and independent one-dimensional cycles, concatenate these counts with inexpensive graph summary statistics, and process the resulting sequence with a small Transformer. The representation should retain multiscale structural information while avoiding pairwise persistence-diagram…
Useful7/10
Difficulty6/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Replace unconstrained softmax cross-attention with a many-to-many transport matrix whose row and column masses have explicit upper bounds. Compute the attention cost from both feature similarity and pairwise relational disagreement, so a token is attended to only when its relationships to other tokens are jointly compatible. The inequality constraints provide a principled dustbin-free mechanism for ignoring distractor tokens.
Useful7/10
Difficulty6/10
Novelty5/10
✓✓ Beats tuned baseline
2026
Replace a dense mixing or attention matrix on tokens arranged on a Cartesian grid by a product of learned or fixed one-dimensional concentration operators. The layer applies one axis operator at a time, reducing parameter and compute cost while enforcing that the global operator is a positive contraction with controlled spectral leakage.
Useful7/10
Difficulty5/10
Novelty5/10
✓✓ Beats tuned baseline
2026
Replace heuristic graph positional encodings with exact finite-abelian-group coordinates derived from edge-class increments and cycle constraints. Relative positions become group differences, allowing a graph transformer to share parameters across repeated generator displacements while retaining exact path consistency and compact cyclic coordinates.
Useful7/10
Difficulty6/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Replace a dense graph-attention or token-mixing matrix by a resolvent-like interaction operator and truncate it to graph neighborhoods whose radius is selected from an estimated spectral gap. Unlike fixed-window sparse attention, the sparsity level is tied to a measurable stability parameter and has an explicit exponential tail criterion.
Useful7/10
Difficulty6/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Use the quotient group's generator classes as a finite relation vocabulary and tie message functions by group displacement instead of by individual graph edges. This creates a compact, exactly consistent relation-aware GNN that can recognize repeated local structure and transfer parameters across graph instances sharing the same Cayley geometry.
Useful6/10
Difficulty5/10
Novelty8/10
✓✓ Beats tuned baseline
2026
Construct a sparse attention or message-passing graph as a gain graph, where every directed edge carries a relative frame label and reverse edges carry the inverse label. Grow the graph using the paper's 2-extension operation: replace two old edges by a new vertex connected to their four endpoints, while preserving the relative gain products. The resulting mask is intended to preserve global information flow under controlled sparsity and to avoid isolated components and brittle bridges commonly…
Useful6/10
Difficulty6/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Add a fixed or weakly parameterized residual mixer whose interaction between sequence positions at distance \(r\) is proportional to \(1/(r\log^2 r)\). Instead of truncating the kernel at a short radius, represent its heavy tail with dyadic distance bands and compute each band using prefix sums or block pooling, giving every token access to arbitrarily distant context at roughly \(O(L\log L)\) cost.
Useful6/10
Difficulty5/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Add a differentiable hypergraph layer that converts invariant edge-length features into triangle areas or higher-dimensional simplex volumes before message passing. Select or weight simplices according to the singular values of the length-to-volume Jacobian, so the network receives geometrically independent features rather than many redundant or nearly degenerate measurements.
Useful6/10
Difficulty5/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Replace ordinary additive path aggregation in graph attention with ordered products of edge operators equipped with learned reversal and color-switch maps. Closed-loop products become a consistency signal, allowing the model to retain direction-sensitive relational information that standard permutation-invariant message passing can lose.
Useful6/10
Difficulty5/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Estimate the temporal spectrum of each sequence channel using a locally private procedure, then apply a regularized inverse-square-root spectral filter before the sequence enters attention or an SSM. The filter removes predictable low-frequency or narrow-band redundancy while avoiding unstable amplification at frequencies where the private estimate is small.
Useful6/10
Difficulty5/10
Novelty7/10