Unverified
2026
Replace an unconstrained attention or MoE routing matrix with a row-stochastic matrix that preserves a learned Gibbs-like distribution over groups. Anneal its temperature so that the router converges to a block-upper-triangular operator: mixing remains flexible within equal-energy groups, while cross-group traffic becomes directional rather than oscillating or collapsing. This creates a hierarchy-aware inductive bias and can reduce the number of active expert or attention connections.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Add a consistency loss for a neural model processing irregularly sampled functions or sequences, comparing its latent covariance representation before and after a time-coordinate warp. The target discrepancy should be near zero for affine transformations but should grow with the normalized curvature, bandwidth, and local sparsity of a nonlinear warp.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace heuristic softmax load balancing with a convex entropic projection that computes expert assignment fractions satisfying explicit linear capacity and load constraints. The projection is bounded componentwise and keeps poorly identified routing decisions away from extreme 0/1 allocations, reducing expert collapse and overload under noisy or shifted batches.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Represent an intermediate hidden-state matrix as a low-rank factorization and replace a dense residual update by a repeated contractive step followed by hard SVD truncation. Increase or decrease the rank automatically so that the discarded Frobenius-norm tail is below a specified tolerance, providing an explicit accuracy-memory tradeoff instead of fixing the rank in advance.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Construct each hierarchy transition as a coarse-graining map together with a local decoder that recovers the retained local representation. This creates a reversible-information path for features needed by later computation while still allowing high-frequency or locally irrelevant components to be discarded.
Useful6/10
Difficulty6/10
Novelty5/10
Unverified
2026
Use a small set of known or trusted pairwise costs to estimate the effective entropic temperature of a Sinkhorn attention or mixture-of-experts routing layer directly from its observed transport plan. This provides a calibration controller that can detect over-concentrated routing and adjust epsilon without backpropagating through a costly temperature search.
Useful6/10
Difficulty4/10
Novelty8/10
Unverified
2026
Replace unconstrained token-to-expert routing with a nonnegative mixture of a small set of feasible routing or communication patterns. Enforce resource limits using the paper's one-sided positive-contribution bound, which gives a conservative certificate without enumerating all joint token activation scenarios. Expand the pattern set only when a separation procedure finds a routing pattern that improves the router objective while adding useful capacity information.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Add a recurrent associative matrix to each selected transformer layer so recent key-value relationships can be retrieved without retaining every past token or performing gradient updates. The matrix uses input-dependent retention and write gates, but retrieval is always performed from the pre-write state, preventing the current target from leaking into its own prediction. Frobenius-norm clipping makes the recurrent memory bounded and provides a direct stability control.
Useful6/10
Difficulty4/10
Novelty4/10
Unverified
2026
Replace an unconstrained residual block by a first-order gradient-flow correction whose energy contains first-, second-, and third-difference penalties, mirroring the paper's higher-gradient gravitational energy. The correction suppresses high-frequency modes while retaining a trainable nonlinear residual branch, and its step size can be chosen from an explicit spectral stability bound.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace an unconstrained hidden-to-hidden interaction in an MLP or transformer feed-forward block by two gauge-related branches. Split channels with an orthogonal involution Θ, constrain the learned interaction K to anticommute with Θ, and use opposite signs of K in paired branches. This creates a testable inductive bias in which the learned interaction only transfers information between the two channel subspaces.
Useful6/10
Difficulty5/10
Novelty8/10
Unverified
2026
Replace a dense token-mixing, MLP, or low-rank-adapter projection with a Bernoulli-signed sparse matrix normalized by the paper's predicted operator scale. Estimate the active representation dimension and use the effective scale to detect regimes in which extreme sparsity is likely to cause unstable amplification or dead rows.
Useful6/10
Difficulty4/10
Novelty5/10
Unverified
2026
Replace an ordinary local convolution or token-mixing block by two nonnegative feature populations A and B that diffuse and drift along the spatial or token axis, with transport rates increasing quadratically with local population and with an optional directional bias. Add a local A plus B to empty reaction so mutually conflicting feature mass is removed rather than merely averaged. The module should produce adaptive competition, and its isolated relaxation should exhibit a measurable t raised…
Useful6/10
Difficulty5/10
Novelty8/10
Unverified
2026
Impose a screened pair-correlation dynamics on stochastic neural replicas so that correlation fluctuations relax locally instead of propagating across the entire representation. The key control knob is a learned or scheduled relaxation rate \(\mu_{FB}\), which predicts a measurable correlation length \(\xi_{FB}=\sqrt{D_{eff}/\mu_{FB}}\). This can be used as a locality regularizer for token representations, diffusion trajectories, or recurrent hidden states.
Useful6/10
Difficulty5/10
Novelty8/10
Unverified
2026
Replace the raw position/channel basis of a one-dimensional sequence module by eigenvectors of the projected cubic radial kernel matrix. Penalize or truncate coefficients in eigenmodes with many sign changes, giving a mathematically ordered smooth-to-oscillatory inductive bias while preserving the two-dimensional nullspace corresponding to affine trends.
Useful6/10
Difficulty4/10
Novelty6/10
Unverified
2026
Replace the linear phase progression in a positional encoding or rotary attention mechanism with a deterministic quadratic phase. The resulting position signal is generated by an irrational rotation with linearly changing increments, and the paper proves that its infinite diffraction measure is purely absolutely continuous, suggesting disorder-like spectral coverage without random sampling.
Useful6/10
Difficulty4/10
Novelty6/10
Unverified
2026
Partition neural modules into two empirically identified reliability or noise classes and restrict their communication graph to a two-block stochastic block model. Allocate a fixed connectivity budget across within-class and cross-class edges using a water-filling update that favors block pairs producing the largest increase in validation utility. The resulting layer is sparse and modular, with a testable prediction that optimal connectivity concentrates on a few block pairs rather than…
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace fixed LoRA factors with a rank-adaptive moving subspace whose columns are augmented using derivative information from several Runge–Kutta stages. The optimizer integrates a matrix-valued gradient-flow approximation inside this enlarged left/right basis, allowing high-order motion of the adapter subspace while retaining a low-rank parameterization.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Add a local Jacobian spectral regularizer and an initialization sweep to steer a looped transformer away from uncontrolled near-unit dynamics. The goal is to prevent examples from entering a fold-critical regime with very long relaxation times, or alternatively to deliberately target a controlled critical regime when adaptive test-time compute is useful.
Useful6/10
Difficulty6/10
Novelty5/10
Unverified
2026
Add a small dynamical state on the transformer module graph and use it to control adaptive computation, but reject controller parameters whose discrete-time update has latent roots outside the unit disk. The state can modulate halting thresholds, residual-block gains, and memory gates; the certificate applies to the controller integrator and prevents unstable oscillations or exploding internal control signals during long adaptive-depth rollouts.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Construct a residual neural network or graph message-passing layer whose skip edges are sampled with probability proportional to their distance as $|i-j|^{-(1+\sigma)}$, while retaining a small local backbone. The paper's mechanism predicts that coarse-grained propagation is governed by the long-range kinetic operator and is therefore asymptotically insensitive to the particular Bernoulli graph realization, yielding controllable superdiffusive information transport without dense all-to-all…
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Model dynamic routing as a multitype branching process: an active token of type d probabilistically creates child activations of type d'. Estimate the corresponding mean offspring operator and regulate its Perron root to a target reproduction rate, typically near one. This should make adaptive-depth or recursively routed networks use sparse computation without producing either rapidly vanishing paths or uncontrolled activation explosions.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Represent each token value in an attention head as a bounded three-dimensional Lie-algebra vector and impose a weighted polygon-closure condition on the aggregate value vectors. The module is invariant to a common \(SU(2)\simeq SO(3)\) rotation of all token vectors, preventing the head from spending capacity on an arbitrary global orientation. A soft closure penalty gives a drop-in experiment, while projection onto the zero-sum manifold provides a harder constrained variant.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace unrestricted global attention or purely local convolution by a sparse distance-dependent interaction graph on a two-dimensional feature map. The edge probability or attention prior decays as \(r^{-(2+\sigma)}\), and \(\sigma\) becomes an explicit architectural control knob: small \(\sigma\) supplies mean-field-like global mixing, intermediate \(\sigma\) supplies long-range Wilson–Fisher behavior, and \(\sigma>2\) approaches a short-range model. The architecture should be evaluated not…
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace an unconstrained Fourier-domain linear mixer with a bank of positive spectral kernels and a max-times erosion aggregator. For a nonnegative Fourier magnitude f, each kernel produces a quotient response f/psi_k and the layer takes the pointwise supremum over kernels, giving exact positive homogeneity and monotonicity. This is most suitable as a drop-in spectral mixing block in a CNN, vision transformer, or state-space model.
Useful6/10
Difficulty5/10
Novelty7/10