Unverified
2026
Constrain a sparse attention graph to split into a k-degenerate backbone and a residual graph with maximum degree at most k-1. Orient the backbone according to a degeneracy order so that each token receives or emits at most k structured interactions in the relevant direction, while the residual edges form a bounded-degree correction layer. This replaces arbitrary sparse attention with a topology that is easier to schedule and whose worst-case edge and local-degree costs are explicit.
Useful5/10
Difficulty6/10
Novelty8/10
Unverified
2026
Replace a costly global PSD constraint on a learned symmetric similarity or covariance matrix with the paper's 2-local PSD constraint. Every 2-by-2 principal submatrix is guaranteed valid, preventing excessively large pairwise correlations while avoiding eigendecomposition or Cholesky factorization of the full matrix.
Useful5/10
Difficulty4/10
Novelty7/10
Unverified
2026
Represent a learned sparse attention or routing pattern as a graph and penalize its second-moment defect, which measures distance from a shifted family and therefore from nested, threshold-like neighborhoods. At inference, optionally replace the learned mask by a nearby shifted mask to obtain more structured sparse indexing and predictable routing patterns.
Useful5/10
Difficulty6/10
Novelty9/10
Unverified
2026
Replace a complex latent vector x in C^d by squared magnitudes of m learned complex linear projections. Set m equal to 2d: the paper proves that m less than or equal to 2d minus 1 cannot generically preserve the latent up to global phase, whereas m equal to 2d is generically sufficient, giving a principled minimal width for a phase-invariant neural bottleneck.
Useful5/10
Difficulty4/10
Novelty7/10
Unverified
2026
Convert attention scores into binary incidence set systems at one or several score thresholds, then group queries that induce the same sampled-key pattern. Compute the expensive key-value aggregation once per pattern and reuse it for all queries in the group. The method is exact for the thresholded routing component and approximates dense attention when queries have a small number of stable high-weight neighborhoods.
Useful5/10
Difficulty6/10
Novelty7/10
Unverified
2026
For a neural model whose outputs lie on a d-dimensional analytic family in a very high-dimensional space, replace the full output vector by 2d+1 or a modestly oversampled number of fixed Gaussian scalar measurements. The paper's theorem predicts almost-sure injectivity in the noiseless setting, so an inverse network or decoder can recover the same latent instance without processing the full observation. Because the theorem does not provide a noise-stability constant, use M=4d+8 or M=8d in the…
Useful5/10
Difficulty4/10
Novelty5/10
Unverified
2026
Replace or augment the transition map of a recurrent state-space model with bounded analytic maps of a latent complex coordinate, using several finite Blaschke generators that share a fixed point. Enforcing a superattracting fixed point of local degree p creates a tunable hierarchy of memory erasure: the theory predicts double-exponential decorrelation with exponent log p, while a merely attracting fixed point gives ordinary exponential decay.
Useful5/10
Difficulty7/10
Novelty9/10
Unverified
2026
Construct a sparse token-mixing architecture by interleaving learned per-token transformations with fixed perfect-shuffle and cyclic-pile permutations. For n not a power of k, the generated permutation group is 2-transitive, so sufficiently rich sequences of generator words can expose every ordered token pair without constructing a dense N by N attention matrix.
Useful5/10
Difficulty4/10
Novelty6/10
Unverified
2026
Replace an unrestricted additive recurrent or fast-weight memory with a sign-selectable update: for each incoming update vector, choose between adding and subtracting it so that a smooth compact potential of the memory state is minimized. This is appropriate when the memory representation has sign symmetry, such as signed random features or a learned linear sketch; it is not a drop-in replacement for ordinary gradient updates where the sign carries semantic information.
Useful5/10
Difficulty5/10
Novelty9/10
Unverified
2026
Replace an unconstrained geometric multiscale codebook by features generated from a finite digit set and a Pisot scale factor. The contracting algebraic-conjugate directions should suppress near-collisions between representations at different scales, producing a discretely separated hierarchy that can be used for embeddings, recurrent memory, or quantized transformer states.
Useful5/10
Difficulty6/10
Novelty9/10
Unverified
2026
Use the sharp exponential tail bound to set a local clipping threshold from a desired exceedance probability. Instead of globally clipping activations at a fixed value or percentile, clip each local window at its minimum plus B log(e/delta), where delta is the tolerated fraction of clipped entries.
Useful5/10
Difficulty3/10
Novelty7/10
Unverified
2026
Replace unconstrained recurrent-state decay with a one-dimensional latent defect field whose states evolve by local diffusion and pair reactions. Defects can move over long distances and persist, while creation and removal occur only in pairs, giving the memory a structured cancellation mechanism that is potentially better suited to delayed-event and parity-like sequence dependencies than a standard GRU or diagonal SSM.
Useful5/10
Difficulty5/10
Novelty8/10
Unverified
2026
Apply a weak-trace spectral constraint to the covariance of antisymmetric second-order features, encouraging a 1/i eigenvalue envelope rather than forcing a finite trace norm. This targets the paper's sharp logarithmic Ky Fan behavior and may preserve useful long-tail interaction directions that nuclear-norm regularization would remove.
Useful5/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace dense token-to-token attention with attention over connected token groups, called polymers, while forbidding nearby polymers from being simultaneously selected. Each candidate group receives an exponentially decaying size and boundary penalty, and the layer sums or samples only compatible collections of groups. The construction should create structured sparsity and prevent redundant overlapping attention regions.
Useful5/10
Difficulty6/10
Novelty7/10
Unverified
2026
Parameterize a learned 3-state transition operator as a product of at most seven elementary row-stochastic matrices rather than learning its nine entries independently. Each factor performs one convex pull-in of row i toward row j, so every intermediate and final matrix remains row-stochastic and the layer has a sparse, bounded-depth interpretation.
Useful5/10
Difficulty4/10
Novelty7/10
Unverified
2026
Replace a learned causal mixing profile by a monotone profile followed by a prefix-average Hardy mixer. The monotonicity constraint makes the mixer provably non-degenerate in the BMO sense: localized variation in the profile cannot be reduced below a calibrated factor by prefix averaging. This is a cheap alternative to dense causal attention for tasks where importance or state profiles are expected to decay along sequence position.
Useful5/10
Difficulty4/10
Novelty8/10
Unverified
2026
Assign each of K entity or token types an integer code from a B_{2,\Delta}-set A, so every unordered pair {i,j} produces a unique and margin-separated scalar code a_i+a_j. Use this code as a compact symmetric pair feature for graph edges, attention biases, or pairwise relation MLPs, avoiding collisions that occur when ordinary low-dimensional additive encodings are quantized or hashed.
Useful5/10
Difficulty4/10
Novelty7/10
Unverified
2026
Apply the paper's extremal tail bound to independently sampled nonnegative neural-network contributions, such as stochastic-depth branch activations, independently gated expert loads, or separately allocated memory chunks. Penalize the analytic worst-case probability that their sum exceeds a budget, using the fact that the worst admissible distribution is a sparse Bernoulli spike at the threshold.
Useful5/10
Difficulty4/10
Novelty8/10
Unverified
2026
Replace a dense attention pattern by the exact intersection of a fixed or cheaply computed base graph H and a learned shared-label relation. Two tokens can exchange information only when they are adjacent in H and share at least one of d labels, producing a controllable structured sparsity pattern. The label count d becomes an explicit capacity and compute knob: increasing d enlarges the relation vocabulary without requiring a dense pairwise mask.
Useful5/10
Difficulty6/10
Novelty5/10
Unverified
2026
Regularize hidden representations using the number of metric balls required to cover at least a fixed fraction of minibatch probability mass. The outlier tolerance ignores a controlled fraction of atypical samples, while the resolution parameter makes the penalty explicitly scale-dependent. Combine the penalty with a variance floor or reconstruction term so that reducing geometric dimension does not produce a constant representation.
Useful5/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace a dense channel-mixing matrix in a sequence layer with alternating diagonal propagation and sparse unipotent Stokes jumps. The diagonal part carries independently controlled exponential phases, while the unipotent factors implement cheap residual-like mode conversion without changing determinant or requiring a dense matrix multiply. Constrain the phase magnitudes and jump coefficients during training to obtain a reversible, norm-monitorable mixer.
Useful5/10
Difficulty4/10
Novelty7/10
Unverified
2026
Use the paper's lower bound on nearest-correlation repair cost to detect when a neural network's pairwise similarity predictions contain too much globally incompatible off-diagonal energy. Instead of projecting every predicted matrix onto the correlation cone, train the network to reduce the repair-risk statistic or trigger expensive repair only when a cheap diagnostic predicts substantial distortion.
Useful5/10
Difficulty4/10
Novelty6/10
Unverified
2026
Replace an unconstrained recurrent or state-space transition with a complex-orthogonal flow generated by a skew-transpose matrix. The transition preserves a bilinear quadratic quantity exactly, preventing repeated application across long sequences from causing norm explosion or decay in the linear dynamics.
Useful5/10
Difficulty5/10
Novelty5/10
Unverified
2026
Insert a Walsh-Hadamard transform before a quantized categorical or activation bottleneck and assign coordinate-dependent quantization precision using the attenuation spectrum of a quaternary symmetric noise model. Coordinates corresponding to tensor-product frequencies with many nonzero indices are attenuated by higher powers of \(\delta\), so their quantization can be made coarser with little effect on the reconstructed post-noise representation. This creates a structured, fast transform…
Useful5/10
Difficulty4/10
Novelty6/10