△ Mechanism confirmed, baseline not beaten
2026
Replace the dense hidden-state trajectory of a continuous-depth or recurrent neural block by a rank-r factorization F(t) = X(t) S(t) V(t)^T, and evolve the factors with a reversible projector-splitting integrator. During backpropagation, reconstruct earlier hidden states by reversing the factor updates rather than storing all activations.
Useful8/10
Difficulty7/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace the usual sum of pairwise modality similarities with a higher-order score based on the coordinatewise Hadamard product of all normalized modality embeddings. For modalities indexed by i=1,...,m, score a tuple using s(x_1,...,x_m)=\omega^\top(\bar g_1(x_1)\odot\cdots\odot\bar g_m(x_m)), where \omega is learned and \odot is coordinatewise multiplication. This adds explicit m-way interactions without concatenating raw features or introducing a joint encoder.
Useful8/10
Difficulty4/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace arbitrary directed-edge weights in a graph neural ODE or recurrent message-passing layer by weights constructed to make the directed Laplacian diagonalizable. This removes Jordan-block coupling, allowing the linearized graph dynamics to be represented as independent eigenmodes rather than modes with polynomial transients such as t^k exp(lambda t).
Useful8/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a standard ReLU surrogate with an input convex neural network whose hidden-to-hidden weights are constrained to be nonnegative. The network remains piecewise linear and expressive, but its convexity allows downstream minimization to use continuous ReLU epigraph constraints instead of binary activation variables, potentially eliminating the integrality bottleneck of neural optimization.
Useful8/10
Difficulty5/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a square dense projection in a Transformer or MLP with a trainable recursive butterfly matrix. The layer preserves multiscale channel interactions while constraining every complementary row-column block to rank at most k, reducing parameters and enabling recursive structured matrix-vector products. Unlike an arbitrary sparse layer, the construction has an explicit recursive factorization and a quasi-optimal approximation guarantee among matrices with the same butterfly rank.
Useful8/10
Difficulty6/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Compress the matrix gradient or momentum before applying Muon's polar LMO, and maintain an error residual in the uncompressed gradient space. The residual prevents systematic sign quantization bias from accumulating, unlike error feedback applied after the nonlinear polar/sign operation. This is suitable for distributed training because workers communicate one sign bit per matrix entry while the server still applies a matrix-aware Muon step.
Useful8/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a complete tensor/Kronecker polynomial lift of a graph dynamical system with observables selected only from the support of the interaction graph. The lifted state can then be propagated by a sparse structured linear operator, while the first omitted degree is treated as an explicit residual or learned closure. This gives a graph-aware polynomial state-space layer for neural ODEs, graph RNNs, and world models.
Useful8/10
Difficulty5/10
Novelty8/10
△ Mechanism confirmed, baseline not beaten
2026
Implement causal linear attention in chunks and combine chunk summaries with an associative scan instead of carrying the recurrent state through all chunks sequentially. This preserves the exact causal computation while reducing inter-chunk dependency depth from the number of chunks to its logarithm, enabling substantially more GPU parallelism for long-context training and prefill.
Useful8/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a dense directed attention matrix by a collection of K learned source-to-hub-to-target interactions. Each hub corresponds to a directed biclique, allowing many source tokens to communicate with many target tokens using O(NK) rather than O(N^2) pair interactions. The construction preserves asymmetric information flow and can be initialized from a graph cover of high-attention edges.
Useful8/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Compress each layer's KV tensor with a partial Tucker approximation over token and feature axes, then encode the truncation residual with a rotated uniform quantizer. Select token rank, feature rank, and residual bit-width jointly under a global byte budget, allowing values with flat spectra to receive residual bits while keys may receive more low-rank capacity.
Useful8/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a deep feed-forward block by the fixed point z=phi(Wz+Vx+b), with the recurrent weight W constrained so that the fixed point is unique for every input. The same condition makes forward fixed-point iteration stable and makes implicit differentiation well-conditioned, allowing depth-independent memory usage while providing a measurable spectral failure boundary.
Useful8/10
Difficulty5/10
Novelty4/10
△ Mechanism confirmed, baseline not beaten
2026
Replace dense or single-dilation sparse attention with two sequential sparse attention stages whose offsets form co-prime arithmetic progressions. The first stage mixes tokens separated by multiples of M2, the second by multiples of M1; their composition reaches virtual offsets mM2+nM1, providing many structured long-range interactions from only M1+M2-1 physical offset families. Use causal masking and residual connections so the module can replace a standard transformer attention block without…
Useful7/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
For sparse conditional supports, replace enumeration of all possible four-cycles with a graph-theoretic cycle basis. Construct the bipartite support graph, choose a spanning forest, and penalize one residual for each non-tree edge and its induced fundamental cycle. In log space, every other cycle constraint is a linear combination of these basis constraints, yielding a principled sparse regularizer.
Useful7/10
Difficulty5/10
Novelty8/10
△ Mechanism confirmed, baseline not beaten
2026
Support conjunction queries over multiple roles without explicitly storing a huge tensor of repeated objects. Represent the required higher-order memory through query-dependent contractions, enabling compositional retrieval with memory that scales linearly in the number of objects.
Useful7/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Build a neural operator from frozen ambient mechanism blocks and a geometry-specific algebraic constraint adapter. The adapter parameterizes all outputs in the affine set satisfying sampled linear constraints exactly, so the network never produces boundary-violating states and does not require a penalty coefficient or post-step projection.
Useful7/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a quadratic pairwise attention or graph aggregation kernel with a compact, translation-invariant indefinite kernel approximated by signed random Fourier features. The feature map preserves the kernel's negative spectral mass through a diagonal sign matrix, so the resulting linear-time aggregation can represent similarities that ordinary positive-definite random features cannot.
Useful7/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Apply the paper's compositional PAS idea to recurrent or state-space networks by propagating a polytope of possible hidden states and input perturbations over multiple time blocks. Instead of validating one hidden trajectory at a time, maintain a trusted convex family and re-linearize only when its nonlinear-fidelity tolerance is exceeded. This creates a runtime monitor and adaptive horizon mechanism for long-sequence inference, forecasting, and learned world models.
Useful7/10
Difficulty7/10
Novelty8/10
△ Mechanism confirmed, baseline not beaten
2026
Construct deep or recurrent networks whose layer weights are correlated across depth with a prescribed power-law covariance, rather than either fully tying or fully independently sampling layers. The paper predicts two usable design boundaries: \(\gamma=1/2\) for divergence of correlation-induced fourth moments and \(\gamma=1\) for loss of summable-correlation flatness.
Useful7/10
Difficulty6/10
Novelty8/10
△ Mechanism confirmed, baseline not beaten
2026
Represent a large linear map acting on a Cartesian 3D grid and multiple physical channels as a TT-matrix, while retaining separate TT blocks for channel couplings that have different semantics. Apply the layer by sequential contractions with TT cores rather than materializing a dense matrix or a full 3D convolution kernel. Rank truncation provides an explicit accuracy-versus-memory knob and can be applied after optimizer updates.
Useful7/10
Difficulty5/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Insert channelwise normalization whose mean and variance are pooled over the full sequence, allowing a small-receptive-field convolutional labeler to access global sequence statistics without adding dilated convolutions or attention. Use this only for tasks where labels occur in long runs or depend on coarse global composition; retain per-position normalization for tasks requiring strict locality.
Useful7/10
Difficulty3/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a fixed-size learned latent for an array-valued complex tensor with a variable-length list of continuous rank-one spectral atoms. An encoder predicts candidate receive direction, transmit direction, residual off-grid offsets, and complex gains; the decoder reconstructs the tensor analytically from the array-response formula, so changing the antenna dimensions does not require changing the decoder weights.
Useful7/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace an unconstrained trajectory or density network with a stack of RealNVP-style triangular coupling layers whose inverse and log-volume change are analytic. Condition the coupling subnetworks on the task prompt and time, so the same invertible module represents task-specific population states while providing an exactly computable density and score surrogate.
Useful7/10
Difficulty5/10
Novelty4/10
△ Mechanism confirmed, baseline not beaten
2026
Represent a batch of token or feature directions as columns of a matrix X, and construct a complementary feature basis Y whose columns are annihilated by X under a diagonal gauge. Use Y as a second algebraically complementary channel for attention or token mixing, either replacing redundant feature projections or regularizing them toward an exact nullspace relation.
Useful7/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Add sparse directed coupling between parallel neural modules, recurrent states, or distributed replicas so that each module is driven toward a common trajectory without forcing an undirected or balanced communication graph. Select n-1 directed paths per strongly connected component and assign gains using the estimated Lipschitz bound of the uncoupled module; activate the coupling only when its graph-certified strength exceeds the predicted synchronization threshold.
Useful7/10
Difficulty5/10
Novelty7/10