Unverified
2026
Replace a dense node-to-landmark graph-attention or message-passing relation by a dictionary of distinct landmark-neighborhood signatures. Nodes sharing the same signature reuse the same structural landmark aggregate, while their individual hidden states are still passed through the output MLP, preserving node-specific predictions. On bounded-treewidth graphs the number of distinct signatures is provably linear in the number k of landmarks, with an explicit dependence on treewidth.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Construct attention groups as hyperedges of a linear r-uniform hypergraph: every pair of tokens is allowed to share at most one group, while each token participates in approximately the same number of groups. Apply local attention inside each group and aggregate the outputs across groups. The construction inherits the paper's sharp capacity bound and prevents both redundant pair interactions and high-degree token hubs.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Represent a nonnegative attention or routing score matrix by two nonnegative vectors, X = uv^T, and learn only entries on a sparse bipartite graph of important query-key or token-expert interactions. Complete the remaining entries multiplicatively and monitor cycle residuals as a certificate of whether the sparse representation is compatible with rank one. Use local ratio violations to trigger additional edges or relax the rank-one approximation only where needed.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace forward activation dropout in a deep residual network with an unbiased multiplicative mask applied only to gradients entering each residual unit or branch. The forward representation remains deterministic for a given minibatch, while the update receives zero-mean stochastic perturbations that are predicted to reproduce dropout-like limiting dynamics in large-width, large-depth ResNets.
Useful6/10
Difficulty4/10
Novelty7/10
Unverified
2026
Replace a learned embedding for each integer with a compositional embedding of its variable-length Gray codeword. Add an auxiliary constraint that numerically adjacent values have nearby representations, while preserving the ordinary task loss so that the model can learn when numerical adjacency matters.
Useful6/10
Difficulty4/10
Novelty8/10
Unverified
2026
Replace a dense channel-mixing matrix by a sparse complex generalised weighing matrix W with exactly w nonzero entries in every row and column, then use U=W divided by square root of w as a norm-preserving mixer. Restricting to k=2 gives a real matrix with entries in {+1,-1}; k=4 supports signed phase rotations. The exact isometry should preserve signal and gradient norms while reducing channel-mixing cost.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace an unconstrained relative-position attention-bias table by a learnable two-dimensional Fourier representation, or regularize the bias toward a small Fourier ratio. The ratio favors coefficient concentration without depending on the absolute scale of the bias, allowing the trained bias to be reconstructed from a small number of dominant frequencies.
Useful6/10
Difficulty4/10
Novelty7/10
Unverified
2026
Compress a trained graph-attention model's edge-dependent logits or gates in the line-graph Fourier domain. Smooth edge values become concentrated in low-frequency coefficients, allowing low-frequency coefficients to retain more precision while high-frequency residuals use fewer bits or are discarded.
Useful6/10
Difficulty6/10
Novelty8/10
Unverified
2026
Replace one independently learned vector per graph edge with a truncated spectral expansion on the line graph. The model learns coefficients for low-frequency edge modes and reconstructs edge features before message passing, reducing parameters while imposing an inductive bias that incident edges should have correlated behavior.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace a single recurrent state with two coupled one-dimensional latent chains whose relative alignment is periodically shifted during inference. Ferromagnetic coupling preserves locally coherent patterns, while controlled sliding produces a nonequilibrium friction effect that can make global magnetization substantially longer-lived than in a static noisy chain. The shift velocity acts as a measurable memory-control parameter rather than an unconstrained architectural hyperparameter.
Useful6/10
Difficulty6/10
Novelty8/10
Unverified
2026
Build a sparse graph by thresholding normalized token or item inner products, then use the leading eigenvectors of its centered adjacency matrix as geometric features or a low-rank attention-logit bias. The graph avoids storing all pairwise similarities, while the paper's spectral bound supplies a concrete signal-to-noise test for deciding whether the resulting embedding is trustworthy.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace a purely recurrent or state-space history summary with two explicitly separated paths: a fixed-size state channel for compressed sequence mixing and a query-dependent indexed channel for exact or near-exact retrieval. Train a lightweight gate to invoke top-k retrieval only when the recurrent state has insufficient evidence for the current query, preserving near-constant cost on ordinary tokens while preventing catastrophic failures on long-range exact-recall tasks.
Useful6/10
Difficulty5/10
Novelty4/10
Unverified
2026
Replace a dense learned linear operator on continuous or image features by a truncated Hermite projection expansion whose coefficients are directly regularized in a Schatten-p norm. The layer becomes a structured low-rank operator, while the radial Hermite-Laguerre correspondence provides an analytically tractable parameterization and an exact spectral penalty.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Use the paper's norm-regularized conic dualization to impose PSD or SOS-style certificate constraints during neural-network training without forming Schur-complement or second-order-cone liftings. A neural dynamics model can be trained jointly with a polynomial Lyapunov or energy certificate, while the certificate subproblem is solved through accelerated updates in equality-constraint dual variables.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace the linear state transition in a small recurrent or state-space module by a circulant matrix acting on a vector over a finite field. The hidden state then has only finitely many possible values and follows an exactly periodic orbit after at most \(q^n\) states, eliminating numerical drift on modular-counting and symbolic-memory tasks. A learned real-valued encoder and decoder can surround the discrete core, while the transition itself is fixed, searched, or trained with a…
Useful6/10
Difficulty6/10
Novelty8/10
Unverified
2026
Replace or augment an RNN or state-space model hidden state with coordinates on a bounded 3-step nilpotent group. The first layer stores ordinary features, the second layer stores pairwise commutator memory, and the third layer stores nested commutators that can preserve three-time dependencies invisible to first- and second-order summaries. Layered reduction keeps the state bounded while retaining the algebraic interaction structure.
Useful6/10
Difficulty7/10
Novelty8/10
Unverified
2026
Replace a dense token-mixing matrix in a sequence model with a fixed or learnable SBP derivative operator D=P^{-1}Q. The discrete integration-by-parts identity makes the interior mixing energy-neutral or boundary-dissipative, reducing exploding activations in deep residual stacks while preserving directional information along the sequence.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Split a recurrent or state-space model into a persistent slow state and a fast internal state. Every r recurrent steps, preserve the slow state but reset or contract the fast state toward a learned reference, reproducing selective restart rather than a destructive global reset. The expected benefit is suppression of long-range oscillatory and error correlations while retaining trajectory-level information.
Useful6/10
Difficulty4/10
Novelty7/10
Unverified
2026
Train a square orthogonal neural mixer while maximizing its entrywise fourth-power concentration. When optimization reaches a non-permutation stationary configuration, explicitly test rank-two row or column rotations and take a rotation with positive exact second variation, using the paper's constructive saddle-escape mechanism.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace a dense spatial parameter field in a neural field or convolutional adapter by a truncated squared-exponential KL expansion with analytic Gaussian-Hermite modes. The amplitude and correlation length remain trainable, but changing them only rescales coefficients and basis parameters instead of triggering a numerical eigensolve.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace a wide activation vector or spatial feature field by a small set of weighted coordinates that preserves the p-norm of every activation in a learned low-dimensional subspace. Unlike ordinary pruning, the selection objective is uniform over the whole coefficient sphere, so the compressed representation is designed to preserve unseen linear combinations and not merely the training examples.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Use the paper's explicitly solved SU(2)-based extremal flow as a structured recurrent transition instead of learning an unconstrained dense recurrent matrix. The transition has only two scalar parameters, a radius/frequency r and phase phi, while its rotating coefficient pattern continuously mixes four real state coordinates and can be integrated with a norm-preserving Cayley transform.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Use stochastic Frank–Wolfe to train a neural submodule whose parameters lie in a convex feasible region without expensive Euclidean projection. The entropic robust objective supplies the stochastic gradient, while a linear minimization oracle enforces constraints such as simplex mixture weights, an l1 budget, or bounded adapter coefficients.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Parameterize a cell-complex neural network by features on p-cells and derive lower-dimensional boundary features using the cellular boundary map over F2. For a 2D square complex, neighboring plaquette bits determine each link feature through XOR, reproducing the paper's exact gauge-law reconstruction and preventing the network from representing inconsistent open boundary configurations.
Useful6/10
Difficulty5/10
Novelty7/10