Unverified
2026
Constrain a sparse attention graph to split into a k-degenerate backbone and a residual graph with maximum degree at most k-1. Orient the backbone according to a degeneracy order so that each token receives or emits at most k structured interactions in the relevant direction, while the residual edges form a bounded-degree correction layer. This replaces arbitrary sparse attention with a topology that is easier to schedule and whose worst-case edge and local-degree costs are explicit.
Useful5/10
Difficulty6/10
Novelty8/10
Unverified
2026
Convert attention scores into binary incidence set systems at one or several score thresholds, then group queries that induce the same sampled-key pattern. Compute the expensive key-value aggregation once per pattern and reuse it for all queries in the group. The method is exact for the thresholded routing component and approximates dense attention when queries have a small number of stable high-weight neighborhoods.
Useful5/10
Difficulty6/10
Novelty7/10
Unverified
2026
Group neural features into small Hermitian matrix elements and scale each group with the paper's tracial spectral Lp norm rather than independently normalizing scalar channels. This introduces a coupled spectral geometry while remaining implementable with ordinary eigendecompositions in the associative Hermitian-matrix special case.
Useful5/10
Difficulty5/10
Novelty8/10
Unverified
2026
Add a structured token-mixing layer based on commuting sums of swap operators rather than unconstrained pairwise attention. The layer learns a low-degree spectral filter in the Jucys–Murphy operators, allowing it to represent hierarchical interactions while retaining an explicit algebraic inductive bias.
Useful5/10
Difficulty6/10
Novelty7/10
Unverified
2026
Adapt the slope of each spiking neuron's surrogate derivative using the normalized entropy of its block's attention distribution. High centered entropy uncertainty increases the slope, while low uncertainty decreases it, and a dead zone holds the default slope fixed for ordinary fluctuations. The adaptation exists only in backpropagation, so the forward spike function, parameter count, and inference cost remain unchanged.
Useful5/10
Difficulty4/10
Novelty5/10
Unverified
2026
Regularize a learned two-dimensional score or value surface so that every local rhombus obeys the hive inequalities. This imposes discrete concavity along three lattice directions, encouraging smooth but nontrivial piecewise-linear structure without simply penalizing all second derivatives.
Useful5/10
Difficulty3/10
Novelty6/10
Unverified
2026
Use the flat-torus covariance bound as a representation regularizer that controls the largest covariance eigenvalue while maintaining a prescribed total variance. This creates a directional anti-collapse constraint rather than only a scalar variance penalty, and can be applied to encoder outputs, VAE latents, or Transformer sequence representations.
Useful5/10
Difficulty3/10
Novelty4/10
Unverified
2026
Construct a sparse token-mixing architecture by interleaving learned per-token transformations with fixed perfect-shuffle and cyclic-pile permutations. For n not a power of k, the generated permutation group is 2-transitive, so sufficiently rich sequences of generator words can expose every ordered token pair without constructing a dense N by N attention matrix.
Useful5/10
Difficulty4/10
Novelty6/10
Unverified
2026
Use the normalized determinant of a routing or attention interaction matrix as a global spectral signature. Penalize abrupt changes in this Laurent-polynomial signature when the model learns or dynamically rewires its interaction graph, preserving global connectivity patterns while still allowing local edge adaptation.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Encode each scalar or discrete code t by the strictly convex lift γ(t)=(t,t²), optionally followed by a learned linear projection and normalization. Because three-code sums on this curve have only near-minimal additive energy, the representation should produce fewer collisions when a model composes three tokens, codes, or retrieved items by addition.
Useful5/10
Difficulty3/10
Novelty7/10
Unverified
2026
Regularize a spatiotemporal neural model with spectral penalties corresponding to several temporal-spatial scaling laws rather than using a single isotropic smoothness penalty. The model can remain spatially detailed while suppressing temporal oscillations, or learn the opposite preference when the data demand it.
Useful5/10
Difficulty4/10
Novelty6/10
Unverified
2026
Replace an unconstrained geometric multiscale codebook by features generated from a finite digit set and a Pisot scale factor. The contracting algebraic-conjugate directions should suppress near-collisions between representations at different scales, producing a discretely separated hierarchy that can be used for embeddings, recurrent memory, or quantized transformer states.
Useful5/10
Difficulty6/10
Novelty9/10
Unverified
2026
Train attention logits so that the associated Sinkhorn-scaled operator has a favorable local spectral gap, making iterative normalization contract faster. Add a differentiable penalty on the second eigenvalue of the normalized operator while retaining the task loss and marginal-feasibility loss.
Useful5/10
Difficulty6/10
Novelty6/10
Unverified
2026
Use the sharp exponential tail bound to set a local clipping threshold from a desired exceedance probability. Instead of globally clipping activations at a fixed value or percentile, clip each local window at its minimum plus B log(e/delta), where delta is the tolerated fraction of clipped entries.
Useful5/10
Difficulty3/10
Novelty7/10
Unverified
2026
Regularize hidden activations or attention logits by their local mean excess above the local minimum, rather than by symmetric variance or absolute magnitude. The penalty specifically suppresses upper-tail spikes while remaining invariant to adding a constant offset to every value in a local window.
Useful5/10
Difficulty4/10
Novelty8/10
Unverified
2026
Whiten intermediate feature vectors and constrain several gauge moments to remain in the dimension-dependent interval predicted by the paper's Gaussian/log-concave comparison. Apply the penalty only to moderate orders, where the paper gives a uniform bound independent of the particular log-concave distribution; this should suppress heavy activation tails without forcing all features to be exactly Gaussian.
Useful5/10
Difficulty3/10
Novelty6/10
Unverified
2026
Add a learnable orthogonal rotation to a hidden representation and train it to make every channel projection have a small ψ2/L2 ratio. Unlike variance normalization, this explicitly suppresses directions with unusually heavy empirical tails while preserving the total quadratic energy of the representation.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Add a nonlocal logarithmic-Laplacian penalty to intermediate spatial feature maps or ordered token embeddings. Unlike a standard graph or image Laplacian, the kernel uses scale-free weights proportional to |z|^{-n} and includes a local compensation term, allowing multiscale feature smoothing without simply forcing nearby features to become identical.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Build a differentiable assignment layer whose rows represent tokens and whose columns represent experts, memory slots, or attention slots. Each row has unit probability mass, but no column receives positive mass from two rows; maintaining at least one vacant column makes assignments continuously deformable through elementary vacancy moves instead of abrupt softmax switches.
Useful5/10
Difficulty6/10
Novelty5/10
Unverified
2026
Construct a filtered cell complex from neural activations or a learned token/feature graph and track its persistence barcode incrementally as model activations change. Replace full persistent-homology recomputation at every checkpoint by maintaining homology bases and applying local transpositions when filtration blocks split or merge; use barcode drift as a training monitor or a weak regularization signal.
Useful5/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace a dense attention pattern by the exact intersection of a fixed or cheaply computed base graph H and a learned shared-label relation. Two tokens can exchange information only when they are adjacent in H and share at least one of d labels, producing a controllable structured sparsity pattern. The label count d becomes an explicit capacity and compute knob: increasing d enlarges the relation vocabulary without requiring a dense pairwise mask.
Useful5/10
Difficulty6/10
Novelty5/10
Unverified
2026
Add a measurement-conditioned attention layer with two explicitly separated fields: a geometry-only inverse-temperature profile that controls interaction strength and an outcome-dependent chemical-potential bias. For a region bounded by coordinates a and b, force the interaction gate to vanish as the square root of the distance from either boundary, while allowing a separate potential channel to encode measured values.
Useful5/10
Difficulty4/10
Novelty7/10
Unverified
2026
Use the paper's exponential dressing of an activity coupling as an adaptive gate on a neural network's nonlinear residual branch. The branch is strongly suppressed when the local activation fluctuation variance is high, producing an automatically linearized and more stable update, while low-variance representations preserve the learned nonlinear interaction.
Useful5/10
Difficulty3/10
Novelty6/10
Unverified
2026
Replace part of an attention matrix with a mixture of fuzzy permutation matrices induced by short permutations. Each basis element represents an order-preserving k-token matching smeared over all embeddings into the sequence, while a balancing constraint makes the aggregate attention receive uniform global coverage. Retain a standard low-rank or local-attention residual so the structured branch does not prevent arbitrary content-dependent interactions.
Useful5/10
Difficulty5/10
Novelty7/10