Unverified
2026
Use the number of tokens participating in an attention operation to adapt the projected query/key dimension according to the sharp Johnson–Lindenstrauss bound. Short attention windows receive a dimension limited by n-1, while longer windows use a logarithmic dimension, avoiding a fixed large head dimension when the current attention set contains too few points to require it.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace a long sequence of graph propagation steps used to approximate a spectral graph filter with one block Krylov projection. Construct a basis from the input node features together with a small number of Gaussian probe vectors, evaluate the desired matrix function only on the resulting small projected matrix, and retain the output columns corresponding to the original features.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Add a distributed spectral positional encoding to a graph neural network, graph transformer, sparse-attention model, or MoE router by computing the dominant eigenvector of the current weighted adjacency matrix with a few warm-started power iterations. Unlike a Fiedler-vector feature, this encoding uses only local neighbor aggregation, is naturally nonnegative for nonnegative adjacency weights, and can be updated incrementally when the graph or edge weights change.
Useful6/10
Difficulty4/10
Novelty4/10
Unverified
2026
Replace a generic learned update on a triangular feature lattice by a max-plus octahedron recurrence, optionally softened with log-sum-exp. The layer propagates information between two time slices while preserving the paper's characteristic tropical local consistency, which may provide a parameter-efficient inductive bias for grid reasoning, image patches, or graph layouts.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Insert a hyperspherical adapter that splits an embedding into several unit-sphere blocks, changes the dimension of each block, and recombines them with a synchronized spherical join. Train the adapter to preserve pairwise angular distances, while using the paper's max-distortion composition principle to avoid uncontrolled accumulation of blockwise errors.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Place doubly stochastic stream mixing immediately before a quantizer, activation compressor, or latent bottleneck and jointly optimize task loss with estimated code length. The paper's entropy argument says that this linear mixing cannot increase differential entropy, so it can provide cross-stream representation capacity without an ideal entropy-rate penalty; the entropy bottleneck then learns which feature values deserve bits.
Useful6/10
Difficulty4/10
Novelty4/10
Unverified
2026
Replace an unconstrained deep residual recurrence by a discretized diffusion system over feature or token positions, with trainable source terms and analytically constrained boundary feedback. The state remains nonnegative under nonnegative inputs, while negative boundary gains enforce exponential decay of perturbations and prevent exploding activations in very deep stacks.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace a dense translation-invariant interaction matrix with a positive-definite Toeplitz kernel K_n(e^f) whose log-spectrum is parameterized by a small number of Fourier coefficients with 1/|k| decay. Use the paper's explicit quadratic term as a spectral-volume budget, allowing long-range structure while discouraging uncontrolled determinant growth and ill-conditioning. Subtracting this term from a log-determinant regularizer leaves a residual intended to capture higher-order deviations from…
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Search sparse reservoir wiring in graph space rather than repeatedly testing every candidate with its full nonlinear dynamics. Use graph descriptors to predict validation accuracy and nonlinear feature selectivity, then spend exact simulations on candidates with high predicted performance or high surrogate uncertainty.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace fixed graph message weights with a source-node activity gate that amplifies or suppresses every outgoing message from that node. Use the linearized epidemic growth condition to calibrate the residual propagation strength so that the dominant graph mode is near, but below, an explicitly chosen stability threshold rather than being determined accidentally by the graph spectrum.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Represent adaptive computation or MoE routing as a continuous latent trajectory that crosses convex mode walls, with a Minkowski norm defining computational speed. At a wall, choose the outgoing latent velocity by the same constrained variational rule as billiard reflection, and use the local path-length certificate to detect or prevent pathological accumulation of infinitely many routing events in finite depth.
Useful6/10
Difficulty7/10
Novelty8/10
Unverified
2026
Build neural computation graphs with explicitly phase-budgeted serial and parallel branches, treating serial compositions as SRG products and parallel residual branches as SRG sums. Allocate phase centers theta_i so that every loop or branch aggregate stays away from -1, enabling stability-aware architecture search and constructive control of branch gains.
Useful6/10
Difficulty7/10
Novelty8/10
Unverified
2026
Replace an unconstrained q-way polynomial or tensorized feature layer with separate decomposable and primitive interaction channels. The decomposable channel models interactions explainable as products of lower physical-weight feature blocks, while the primitive channel captures residual factors that cannot be represented by those products. This should reduce redundant high-order parameters and provide a controllable inductive bias for compositional or disentangled representations.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Run Metropolis-Hastings directly on neural architectures modulo permutations of structurally exchangeable hidden units, channels, or experts rather than treating every labelled representation as a distinct architecture. Correct the proposal ratio using representation-orbit sizes, so architectures with many internal symmetries receive the intended posterior mass.
Useful6/10
Difficulty6/10
Novelty5/10
Unverified
2026
Make Laplacian latency attention hardware-friendly by quantizing the exponential decay and row normalization to powers of two. Exponentials and divisions can then be implemented with lookup tables and integer shifts, enabling a low-bit attention kernel for neuromorphic and edge accelerators.
Useful6/10
Difficulty4/10
Novelty7/10
Unverified
2026
Approximate a long hidden-state or key/value trajectory by a small sum of exponential modes, storing only the mode nodes and vector weights instead of every timestep. Recover the modes from a Hankel matrix through an annihilating polynomial, then reconstruct the trajectory or a compact recurrent state during inference.
Useful6/10
Difficulty6/10
Novelty5/10
Unverified
2026
Replace an ordinary hierarchical graph encoder with a finite chain-complex encoder whose learned boundary maps satisfy \(\partial_{k-1}\partial_k=0\). Compute Smith normal form on the integer incidence matrices and treat unit-labelled cell pairs as refinement overhead: cancel or gate those pairs before message passing, while preserving non-unit labels that encode genuinely nontrivial structure. The resulting representation should be insensitive to arbitrary cell subdivision while retaining…
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace a tensor-product network over a low-dimensional state and a large distribution embedding with a neural operator that consumes the distribution vector once and outputs values on a finite-difference grid in the low-dimensional state. Train it with the governing PDE residual, explicit boundary residuals, and optional signed shape constraints, allowing the network to preserve numerical structure that a generic MLP would learn only implicitly.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Replace one-shot top-k expert assignment with a capacity-constrained stochastic routing process in which tokens have a temporary routing direction and can reverse it at rate gamma. Tokens preferentially move through short vacancy clusters, while reversals break persistent directed congestion and should delay or eliminate expert-level jams. This creates a tunable routing phase diagram rather than relying only on an auxiliary load-balancing loss.
Useful6/10
Difficulty6/10
Novelty8/10
Unverified
2026
Replace an unconstrained recurrent reset gate with a threshold policy over hidden-state age and a scalar degradation score. The model continues its recurrence while the estimated cost of retaining the state is below the cost of restarting, then resets and reinitializes the state when the threshold is crossed. This should reduce long-horizon hidden-state drift while using only one scalar gate per sequence position.
Useful6/10
Difficulty4/10
Novelty7/10
Unverified
2026
Add a neural feature, adapter, or expert block only through the component of its outputs that is orthogonal to the span of all previously installed blocks. Quotient coefficient directions that produce nearly identical outputs with an SVD or pseudoinverse, so the new block contributes intrinsic representational dimensions instead of duplicating old features. The expected benefit is a smaller effective architecture and better-conditioned block expansion.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace a freely learned finite impulse-response mixing kernel with a matrix polynomial whose roots are constrained to the unit circle. The resulting block-Toeplitz operator has an explicitly positive semidefinite spectral construction, while increasing the polynomial degree gives a systematic capacity knob for approximating matrix-valued frequency responses.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace independent top-k routing by a tree-structured hypergraph assignment layer. Each candidate route is a singleton or pair of resources, and the router selects exactly q_e routes for every tree edge e while ensuring that no resource is consumed twice. This removes capacity collisions before expert computation instead of repairing them with token dropping or load-balancing penalties.
Useful6/10
Difficulty7/10
Novelty7/10
Unverified
2026
Replace a stationary federated optimizer with a decentralized optimizer whose target distribution explicitly forgets old streaming samples. Each round performs only K consensus-gradient iterations, with K selected from the mixing contraction so that the communication budget matches the temporal volatility of the objective. The method should react faster to distribution shifts while limiting disagreement and bias caused by heterogeneous clients.
Useful6/10
Difficulty5/10
Novelty5/10