Unverified
2026
Replace a large flat positional-embedding table with a recursively decoded nine-way address whose child transformations contract coordinates by exactly 1/3. Encode an input position using features attached to the address prefix at several depths, guaranteeing that increasing depth produces a geometrically localized representation and that an infinite valid address cannot ambiguously represent two distinct points. This is especially suitable for 2D vision tokens, maps, point clouds, or…
Useful5/10
Difficulty4/10
Novelty4/10
Unverified
2026
Build a neural feature-mixing layer from a truncated shift S and a diagonal phase operator T satisfying TS=qST, with |q|=1. The relation forces moving one position in the graded feature basis to multiply the phase operator by q, providing a compact inductive bias for periodic, phase-sensitive, or cyclic data.
Useful5/10
Difficulty4/10
Novelty7/10
Unverified
2026
Add an integer-lattice feasibility layer after ordinary top-1 or top-2 MoE routing. The router first produces its usual expert assignments, then minimally changes a small number of low-confidence assignments so the batch count vector lies in a prescribed lattice or desired coset, eliminating persistent modular load imbalance that ordinary auxiliary losses may not detect.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace raw hyperbolic embedding-radius regularization with a dimension-aware effective-radius target. For embeddings concentrated near hyperbolic radius rho in an n-dimensional hyperbolic space, regulate s times log(sinh(rho) / sqrt(n)) rather than rho itself, and use the same quantity to calibrate distance-logit temperature. This should make hyperbolic metric-learning behavior more invariant when embedding dimension, curvature, or model scale changes.
Useful5/10
Difficulty4/10
Novelty6/10
Unverified
2026
Construct a nested sequence of representations in which each coarser representation is obtained from the previous one by a 1-Lipschitz projection. Train prediction heads at multiple scales so coarse predictions remain stable and approximately recoverable from the finer representation, enabling early exit, token pooling, and controlled multiresolution inference.
Useful5/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace part of dense token mixing with a small bank of structured permutations acting on a hierarchical token tree. Diagonal inheritance shares the same local permutation across all descendant copies, while selector words activate one connector type at a chosen level and remain inactive on the next type, providing controllable multiscale receptive fields without constructing a dense attention matrix.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace a collection of dense task-specific linear layers with a common sparse structural matrix and task-specific edge strengths. All tasks share the same learned connectivity pattern, but retain independent values on active connections, allowing parameter sharing without forcing identical interactions.
Useful5/10
Difficulty5/10
Novelty5/10
Unverified
2026
Treat the maximum dependency distance faithfully modeled by a finite neural architecture as an emergent correlation length, and estimate how it grows with depth, state size, or attention span. Fit the exponent \(\kappa\) and use it as an architecture-selection signal: a model with larger \(\kappa\) should acquire long-range competence more efficiently at equal parameter or FLOP budget.
Useful5/10
Difficulty3/10
Novelty6/10
Unverified
2026
Replace one dense attention layer with a sparse hierarchical attention module whose states are clusters of tokens and whose transitions merge two clusters or reverse a previous merge. Enforce the flag-space cancellation law on pairs of alternative two-step merge paths, so redundant hierarchical routes destructively interfere instead of producing duplicated features. Normalize merge-then-unmerge loops using the product of the sizes of the merged clusters, preventing large clusters from…
Useful5/10
Difficulty7/10
Novelty8/10
Unverified
2026
Represent sequence positions as vertices on a circle and use a maximal family of mutually non-crossing d-arcs to define the allowed attention interactions. Rotate the family by d positions, or use several phase-shifted families across successive layers, so each layer has only O(N) edges but repeated layers propagate information over long distances. The geometric compatibility rule replaces arbitrary local-window or hand-designed sparse masks with a structured family whose maximality gives broad…
Useful5/10
Difficulty4/10
Novelty7/10
Unverified
2026
Replace one-step greedy landmark selection in Nyström attention or kernel compression with a restricted pairwise-lookahead rule. The lookahead is motivated by the paper's explicit obstruction: a signed triangle can make individual column gains exhibit increasing rather than diminishing returns, so the best next column need not belong to the best pair.
Useful5/10
Difficulty5/10
Novelty4/10
Unverified
2026
Replace unconstrained pairwise token-routing interactions with a structured two-token router derived from an involutive set-theoretical Yang–Baxter solution. The pair operator is a convex interpolation between identity and a permutation of discrete routing states, so it cannot amplify probability mass or logits when applied to routing distributions. The Yang–Baxter relation provides a falsifiable test for whether three-token routing updates are insensitive to the two admissible…
Useful5/10
Difficulty6/10
Novelty8/10
Unverified
2026
Construct a neural mixing layer on Fourier or positional modes using a small set of exponentiated Virasoro generators instead of a dense mode-to-mode matrix. The generator coefficients are shared across all inputs, while the Lie bracket fixes how different mode shifts interact; an optional central channel captures the special coupling between modes whose indices sum to zero.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Learn or select sparse cyclic convolution or relative-attention offsets whose pairwise differences collide less often modulo the sequence length. The paper's Fourier fourth-power identity turns this combinatorial objective into an FFT-computable differentiable loss, enabling fixed-K sparse patterns with lower aliasing and interference than random offsets.
Useful5/10
Difficulty4/10
Novelty7/10
Unverified
2026
Construct a shared latent transformation as a neural monad-like operator Γ=Ω∘Σ, and expose its iterates Γ^{q+1}Y as a refinement trajectory rather than stacking unrelated layers. Aggregate the resulting representations with a learned or fixed realization weighting, while training an algebra-action map θ:ΓY→Y to make one-step refinement compatible with the original representation. This creates a shallow-parameter, arbitrarily deep computation path with explicit compositional…
Useful5/10
Difficulty6/10
Novelty6/10
Unverified
2026
Use eigenvector delocalization as a mask-quality criterion rather than selecting a random sparse graph blindly. Penalize masks whose normalized adjacency has concentrated leading eigenvectors or disconnected or weakly connected components, while preserving the power-law distance prior. This creates a sparse routing graph that is less likely to trap information in local regions.
Useful5/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace or augment relative-position attention with a positive fractional-integration mixing kernel whose radial behavior has separate inner and outer power laws. Tokens close to one another interact through the usual fractional singularity, while tokens near different radial scales receive a ground-state correction that can improve multiscale information transport without introducing a dense learned positional table.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace independent top-k expert or attention-edge selection with a gate vector constrained by hierarchical path budgets modeled on the paper's extended-Dyck-path polytope. Ordinary interactions receive continuous nonnegative capacities, while a designated class of cross-group interactions receives binary or clipped-to-one gates, producing structured sparsity and preventing many correlated paths from consuming the same routing budget.
Useful5/10
Difficulty6/10
Novelty7/10
Unverified
2026
Represent MoE experts as leaves of a balanced ternary tree and regularize the hierarchical boundary of each expert's assignment mask. At fixed routing mass x, the ternary martingale isoperimetric theorem supplies the explicit minimum one-variation T_3(x), so the router can be penalized according to an occupancy-dependent profile rather than a uniform parent-child disagreement cost. This should favor coherent, stable routing regions while preventing small expert supports from obtaining…
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Build a neural architecture whose receptive field or attention span is increased according to an estimated disorder-to-order crossover scale. Local branches process windows below the crossover as if they were stochastic, while a global branch is activated only when the context exceeds the predicted scale needed to expose deterministic recurrence. This targets sequences or images containing long-range quasiperiodic, hierarchical, or algorithmically generated structure that is statistically…
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace an unconstrained deep routing tree by a q-ary descendant hierarchy with an explicit even height h=0,2,4,... labeling feature scale or computation depth. Train the router so that empirical occupancy of heights follows the exact even-sector law from the Nagao quotient, preventing concentration at shallow layers or unstable overuse of very deep paths.
Useful5/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace softmax attention or dense MoE routing with a normalized Rayleigh–Jeans distribution over tokens or experts. If an item's energy is close to the chemical potential, its probability becomes disproportionately large, creating controllable low-energy condensation instead of the exponentially smooth allocation produced by softmax.
Useful5/10
Difficulty6/10
Novelty7/10
Unverified
2026
Construct a sparse attention mask from a fixed regular candidate graph and one scalar random label per token, retaining edge $(u,v)$ when $x_u+x_v\geq\tau$. Unlike independent random pruning, this produces correlated neighborhoods and a controllable distribution of token degrees, potentially giving some tokens broad receptive fields while retaining a fixed sparse budget.
Useful5/10
Difficulty3/10
Novelty7/10
Unverified
2026
Replace an unconstrained order-k weight tensor with a sum of components that are symmetric only within selected contiguous index blocks. This preserves interactions between blocks while tying parameters under within-block permutations, providing a tunable middle ground between a fully dense tensor and a fully symmetric tensor.
Useful5/10
Difficulty5/10
Novelty7/10