Unverified
2026
Replace an unconstrained low-rank adapter or similarity projection with a learned subspace carrying a prescribed signed metric. The module learns an orthonormal basis U for a k=p+q dimensional subspace, forces the compressed form U^*I_{m,n}U to have p positive and q negative eigenvalues, and uses the resulting pseudo-inner product for signed attention or retrieval scores.
Useful5/10
Difficulty5/10
Novelty8/10
Unverified
2026
Replace random or learned routing hashes for tokens arranged on a d by d grid with a fixed family of mutually orthogonal anti-Latin squares. Each channel assigns exactly d of the d squared tokens to every bucket, while any two channels jointly distinguish every grid position. The resulting router has deterministic load balance and multi-view positional diversity.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace or augment standard sinusoidal or RoPE position features with bracket-quadratic phases $e(-\theta n\lfloor\beta n\rfloor)$ generated by a Heisenberg nilmanifold orbit. Multiple irrational coefficients and output frequencies produce a cheap deterministic encoding whose empirical cross-position correlations should exhibit cancellation instead of the periodic aliasing of rational or finite-frequency encodings.
Useful5/10
Difficulty3/10
Novelty7/10
Unverified
2026
Construct positional or relative-position features as a nonnegative mixture of lattice cosine functions instead of independently signed sinusoidal features. The resulting bias is the Fourier transform of a positive discrete measure with explicitly bounded spectral support, while the mesh and degree can be initialized in the paper's dense-but-controlled frequency regime.
Useful5/10
Difficulty4/10
Novelty7/10
Unverified
2026
Parameterize a learned token metric as a nonnegative sum of sparse integral rank-one projections with unimodular support, rather than learning an unconstrained dense positive-semidefinite matrix. Graph-incidence covectors give an immediately implementable support family, while nonnegative coefficients guarantee positive semidefiniteness by construction.
Useful5/10
Difficulty5/10
Novelty5/10
Unverified
2026
Replace or augment a scalar periodic positional coordinate with a normalized bank of odd Fourier harmonics, keeping every position on the same-radius sphere. The resulting representation has an explicit translation-invariant similarity kernel, allowing the frequency count and spectral weighting to control how sharply attention distinguishes nearby versus distant phases.
Useful5/10
Difficulty3/10
Novelty3/10
Unverified
2026
Design sparse attention masks using a graph discrepancy criterion rather than selecting only local or nearest-neighbor edges. A mask with approximately uniform edge counts between every pair of token subsets spreads information globally, while the rigidity consequence provides a principled way to preserve enough independent pairwise constraints for latent geometric features.
Useful5/10
Difficulty4/10
Novelty6/10
Unverified
2026
Regularize a learned entrywise attention or graph mask using both its ordinary Schatten-p operator norm and the norm of finite channel-block amplifications. This targets masks that look stable on scalar matrices but become unstable when each token-to-token interaction acts on multi-channel feature blocks.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace an unconstrained three-token interaction block by three distinct pair maps constructed from anticommuting channel generators. For every token triple, enforce equality of the two composition paths A12 B13 C23 and C23 B13 A12, while retaining different parameters for the three edges. This creates a globally consistent three-way interaction without collapsing to a single shared pair operator.
Useful5/10
Difficulty6/10
Novelty9/10
Unverified
2026
Calibrate random edge dropout in a GNN or sparse-attention layer using the spectral radius of the underlying communication graph. Retain edges with probability p chosen so that p lambda(A) is at least 1 plus a safety margin, preventing the random computation graph from entering a subcritical fragmented regime while retaining high sparsity.
Useful5/10
Difficulty4/10
Novelty7/10
Unverified
2026
Replace dense attention on tree-structured inputs with stochastic attention neighborhoods formed by metric balls of sampled radii. Use the paper's exact trimming rule to ensure that every sampled cover remains valid while its total radius budget is bounded, then average predictions over several independent covers during training. This creates sparse, globally covering attention masks with an explicit locality-versus-coverage control.
Useful5/10
Difficulty6/10
Novelty8/10
Unverified
2026
Use the dimension-specific relation A_3=0 to remove all intermediate channels transforming as the third exterior power of the two-dimensional vector representation. In tensor-product attention or equivariant MLPs, this is an exact algebraic pruning rule rather than approximate low-rank compression.
Useful5/10
Difficulty6/10
Novelty7/10
Unverified
2026
Build a binary hierarchy over tokens by recursively splitting each active block with a beta-splitting rule, then perform dense attention only inside small leaf blocks and communicate between leaves through learned summaries at internal nodes. The beta parameter controls how balanced the partition is, while the paper's maximum-depth asymptotic supplies a principled depth budget and a way to detect pathological trees.
Useful5/10
Difficulty6/10
Novelty5/10
Unverified
2026
Add a deterministic feature layer that evaluates symmetric Schur-type features on a fixed cyclic orbit and learned reciprocal latent pairs, then projects the resulting channels onto selected residue classes with an exact roots-of-unity filter. The reciprocal construction makes the layer invariant under replacing each latent scalar by its inverse, while the torsion projector prevents leakage between cyclic frequency sectors.
Useful5/10
Difficulty6/10
Novelty6/10
Unverified
2026
Add an auxiliary loss that makes selected representation coordinates insensitive to all subsets of fewer than d variables while retaining a d-way parity statistic. The objective discourages the network from solving a task through pairwise shortcuts and explicitly rewards a controlled high-order interaction.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Use the paper's eventual path-length bounds to constrain an order-invariant routing graph to a constant-hop communication budget. A learned sparse attention or graph-neural-network layer can explicitly route information through at most three admissible hops, while a more conservative auxiliary route permits at most five minimal-path hops, preventing increasingly long and unstable dependency chains as sequence length grows.
Useful5/10
Difficulty7/10
Novelty8/10
Unverified
2026
Replace dense token-to-token attention on a 2D token grid with local attention plus sparse horizontal and vertical communication axes. Tokens at intersections of selected axes receive extra cross-axis attention edges, creating a reinforced sparse graph that can transmit information across large blocks while using far fewer edges than dense attention. The mask should use light-tailed, approximately geometric spacing in both directions rather than heavy-tailed spacing in one direction.
Useful5/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace independent top-k expert decisions by a global fractional routing problem that enforces token-side and expert-side capacities together with an additional diversity constraint represented by a partition or laminar matroid. Use the resulting Hall-type deficiency certificate to identify overloaded token subsets and penalize the actual structural cause of routing failure rather than relying only on an aggregate load-balancing loss.
Useful5/10
Difficulty6/10
Novelty4/10
Unverified
2026
Use the finite-order characterization to learn a nonlinear similarity function for token, patch, or graph-node Gram matrices while preserving PSD by construction or by a differentiable certificate loss. This creates a kernelized attention or graph-readout mechanism in which nonlinear affinity transformations cannot introduce indefinite similarity geometry.
Useful5/10
Difficulty6/10
Novelty6/10
Unverified
2026
Constrain a sparse attention graph to split into a k-degenerate backbone and a residual graph with maximum degree at most k-1. Orient the backbone according to a degeneracy order so that each token receives or emits at most k structured interactions in the relevant direction, while the residual edges form a bounded-degree correction layer. This replaces arbitrary sparse attention with a topology that is easier to schedule and whose worst-case edge and local-degree costs are explicit.
Useful5/10
Difficulty6/10
Novelty8/10
Unverified
2026
Replace a costly global PSD constraint on a learned symmetric similarity or covariance matrix with the paper's 2-local PSD constraint. Every 2-by-2 principal submatrix is guaranteed valid, preventing excessively large pairwise correlations while avoiding eigendecomposition or Cholesky factorization of the full matrix.
Useful5/10
Difficulty4/10
Novelty7/10
Unverified
2026
Represent a learned sparse attention or routing pattern as a graph and penalize its second-moment defect, which measures distance from a shifted family and therefore from nested, threshold-like neighborhoods. At inference, optionally replace the learned mask by a nearby shifted mask to obtain more structured sparse indexing and predictable routing patterns.
Useful5/10
Difficulty6/10
Novelty9/10
Unverified
2026
Replace pointwise pair interactions between mesh cells by quadrature of the interaction kernel over the full Cartesian product of the two cells. Decompose each cell pair into convex-hull pieces and apply a Duffy-like radial transformation so the coincidence singularity is confined to one quadrature coordinate, allowing fixed Gauss-Jacobi or adaptive quadrature to produce smooth, low-variance interaction features.
Useful5/10
Difficulty7/10
Novelty7/10
Unverified
2026
Apply a trainable scalar gate entrywise to a Min/Max structured affinity or covariance matrix while enforcing that the gate is nonnegative, nondecreasing, and convex. This preserves Loewner ordering on the structured cone and avoids unconstrained elementwise nonlinearities that can destroy PSD or order relations.
Useful5/10
Difficulty4/10
Novelty5/10