△ Mechanism confirmed, baseline not beaten
2026
Replace direct logit gradient updates for a simplex-valued neural module with a cascade consisting of a passive LTI filter followed by softmax. The filter can provide useful memory or momentum, but its transfer function is constrained to remain strictly passive, preventing the destabilization mechanism identified for nonpassive higher-order replicator dynamics.
Useful8/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace an unconstrained deep RoPE attention residual update by a spherical or norm-preserving update whose attention kernel has a known positive floor. Estimate the reversible transverse spectrum of the current attention matrix and choose the residual step size below its explicit Euler stability limit; use the angular token diameter as a runtime contraction monitor.
Useful8/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Replace the usual random or elementwise-positive linear-attention feature map with a rank-one positive-semidefinite feature map derived from query and key vectors. For normalized inputs, the resulting kernel is the squared inner product, which is nonnegative and gives a geometrically structured interference pattern that is better suited to associative recall than an arbitrary low-rank feature map.
Useful8/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace token-by-token KV storage after an SSM or recurrent encoder with an online allocate-on-novelty cache. A new slot is created only when the incoming key is sufficiently dissimilar from every stored key; otherwise the incoming value is merged into its nearest slot, so repeated or redundant content does not grow the cache.
Useful8/10
Difficulty4/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace an isotropic local mixing layer with a kinetic layer that smooths features in x and transports them in y along the characteristic direction x. The layer should be useful for phase-space data, learned simulators, and world models in which positions or transported quantities evolve through coupled drift and diffusion rather than independent Euclidean motion.
Useful7/10
Difficulty5/10
Novelty7/10
✗ Mechanism failed
2026
Replace Euclidean momentum for selected neural parameters with a mirror or Bregman update, while using the paper's accelerated Newton direction for the objective step. Entropy geometry is especially suitable for softmax MoE routers, while Euclidean or log-barrier geometries can be used for unconstrained or positive parameters.
Useful7/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Use a learned scalar ordering function to turn a symmetric local Gaussian graph kernel into a directed, row-stochastic message-passing operator. The asymmetric tilt lets neighboring nodes communicate preferentially along an inferred dynamical direction, while the Gaussian factor retains locality and diffusion-like smoothing.
Useful7/10
Difficulty5/10
Novelty7/10
✗ Mechanism failed
2026
Prune redundant attention heads using separate similarity scores for sink behavior and content routing. Two heads are considered safely redundant only when their normalized content compositions are close in Aitchison distance and their sink-mass trajectories are also close, avoiding pruning decisions dominated by a shared sink token.
Useful7/10
Difficulty4/10
Novelty7/10
✗ Mechanism failed
2026
Distill a teacher's attention into a student by matching sink mass and the normalized content distribution as separate targets rather than applying one KL divergence to the entire attention row. Use the Aitchison distance on the content composition, which compares relative token allocation and prevents a large common sink probability from overwhelming differences between content tokens.
Useful7/10
Difficulty3/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Represent a graph, set, or attributed network as a measurable Z-valued kernel and train on lifted representatives while explicitly minimizing over node couplings. The quotient objective is invariant to relabeling by construction, while the lifted loss gives a dense correspondence signal that can stabilize graph attention and relational encoders.
Useful7/10
Difficulty5/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Replace ordinary per-channel or per-token KV quantization with a structured orthogonal transform followed by blockwise 3-bit quantization. Use a normalized Walsh-Hadamard transform and small SO(4) rotations to spread outliers across coordinates, quantize the transformed vectors, and exploit orthogonality to rotate queries and attention outputs so unquantized attention remains mathematically equivalent.
Useful7/10
Difficulty6/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Build a kernel aggregation layer whose output is a tangent vector field on the unit sphere and whose surface divergence is identically zero by construction. For each source point, use a matrix kernel obtained by applying a surface-rotated gradient in the query variable to a scalar zonal kernel; this is a differential-form version of the paper's matrix-valued construction. The layer can replace attention or message passing when the target dynamics are incompressible, such as spherical fluid…
Useful7/10
Difficulty5/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Replace unconstrained softmax cross-attention with a many-to-many transport matrix whose row and column masses have explicit upper bounds. Compute the attention cost from both feature similarity and pairwise relational disagreement, so a token is attended to only when its relationships to other tokens are jointly compatible. The inequality constraints provide a principled dustbin-free mechanism for ignoring distractor tokens.
Useful7/10
Difficulty6/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Replace or augment dot-product attention with a non-increasing radial kernel of pairwise representation distance. The bandwidth is normalized using an estimated local intrinsic dimension and local neighbor scale, creating an explicit locality-controlled attention operator.
Useful7/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace unconstrained token-mixing logits by a symmetric zero-row-sum response matrix generated from positive conductances on a small auxiliary electrical network. The resulting mixer has conservation and positivity structure, while circular minors have a prescribed sign pattern associated with positive grove measurements. This is especially suitable for graph neural networks and attention variants that need stable global diffusion rather than arbitrary dense affinities.
Useful7/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Replace ordinary pairwise attention similarity by an affinity averaged over transformed keys or values. The resulting attention is invariant to the group action on either input and avoids requiring the network to learn identical attention patterns for every rotated or transformed copy.
Useful7/10
Difficulty5/10
Novelty5/10
✗ Failed on benchmark
2026
Replace dense coarse-to-fine cross-attention at multiresolution interfaces with a sparse, nonnegative overlap operator whose weighted feature average is exactly conserved between the two resolutions. Use this operator as a low-order path and blend it with an unrestricted neural cross-attention path through a convex limiter that keeps features inside a box or simplex domain. The construction is especially suitable for adaptive token grids, hierarchical graph neural networks, neural operators…
Useful7/10
Difficulty5/10
Novelty8/10
✗ Failed on benchmark
2026
Use a smoothed Burg entropy as the mirror map in a proximal-gradient optimizer for positive or simplex-valued neural parameters. The optimizer performs a Bregman-proximal step instead of an additive Euclidean update, while the smoothing parameter avoids the singularity of ordinary Burg entropy at zero.
Useful7/10
Difficulty5/10
Novelty5/10
✗ Failed on benchmark
2026
Use separate measurement-domain and target-domain token sets so a network can infer a field on one spatial domain from sparse observations on another in one forward pass. The same decoder can answer arbitrary target query points, avoiding an optimization loop for each inverse instance.
Useful7/10
Difficulty5/10
Novelty6/10
✓ Mechanism works
2026
Replace an unconstrained nonlinearity on hyperbolic pairwise similarities with a function from the paper's exact Lorentz–Gram preserver family. The transformed similarity matrix remains realizable as Lorentz inner products of future-directed unit timelike vectors, allowing a network to sharpen or smooth hyperbolic neighborhoods without introducing geometrically impossible pairwise relations.
Useful7/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace dense aggregation over patches, points, or graph neighbors with a state-dependent positive quadrature rule. A latent encoder produces the current manifold coordinate z, a small controller chooses a subset of entities, and a convex redistribution step assigns nonnegative weights so that the subset preserves the full aggregation on a learned feature basis. The method is especially suitable for neural operators and vision attention where many spatial entities are redundant along the data…
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Add a consistency loss for a neural model processing irregularly sampled functions or sequences, comparing its latent covariance representation before and after a time-coordinate warp. The target discrepancy should be near zero for affine transformations but should grow with the normalized curvature, bandwidth, and local sparsity of a nonlinear warp.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace an unconstrained positive higher-order gating function by a factorially normalized homogeneous polynomial whose coefficients are trained toward the Lorentzian Hessian signature. This gives multiplicative routing or attention interactions a controlled curvature pattern instead of allowing arbitrary unstable higher-order amplification.
Useful6/10
Difficulty6/10
Novelty8/10
Unverified
2026
Replace unconstrained momentum updates for selected neural-network parameter blocks by damped second-order motion on a unit sphere. The update learns directions while preserving the block norm exactly, and includes the centripetal term required for a valid acceleration constrained to the sphere rather than merely projecting gradients after an unconstrained step.
Useful6/10
Difficulty5/10
Novelty6/10