△ Mechanism confirmed, baseline not beaten
2026
Compress an existing dense neural-network weight matrix into a recursive butterfly operator using Gaussian sketches of complementary blocks. This is useful for deployment or distillation: the dense model provides an oracle for matrix-vector products, while the compressed model stores only recursive transfer bases and small cores. The generalized Nyström identity gives exact reconstruction for rank-k blocks and a principled approximation route for numerically low-rank blocks.
Useful7/10
Difficulty7/10
Novelty6/10
✗ Mechanism failed
2026
Replace fixed PCA-rank selection in a hidden layer with a renormalization-group-inspired gate over covariance eigenvalue bands. The gate retains modes whose effective quartic interaction remains unstable or strongly scale-dependent, while pruning bands that flow toward the Gaussian noise fixed point. Unlike top-eigenvalue truncation, this is designed for extensive-rank signal distributed throughout the bulk spectrum.
Useful7/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Replace deterministic LoRA importance scores with one-sided tests of whether each rank-one update has population contribution at least a user-selected threshold. Maintain empirical contribution samples during fine-tuning, estimate their uncertainty, and prune the components with the weakest statistical evidence while respecting the target rank budget. The method should avoid deleting components merely because their latest minibatch gradient was small.
Useful7/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Construct a neural activation bottleneck by projecting hidden states into a fixed covariance-eigenbasis and retaining only the d largest-magnitude coordinates per sample. For Gaussian, decorrelated activations, the paper proves that adaptive top-d selection in the PCA basis has no greater expected residual energy than adaptive top-d selection after any other orthogonal rotation. This provides a principled alternative to learning an unrestricted rotation before sparsification.
Useful7/10
Difficulty3/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Replace selected dense neural-network operators by low-rank factors whose rank is selected by a randomized residual test at a user-specified tolerance. Construct candidate bases in large blocks for efficient matrix operations, then prune the block to the smallest rank that passes the residual criterion instead of treating the block size as the final rank.
Useful7/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Use the paper's finite-width O(n^{-1/2}) Gaussian-process approximation bound as a width-budgeting rule rather than choosing every hidden dimension uniformly. Estimate an architecture-specific constant for each layer or attention contraction, then allocate width according to the smallest dimension satisfying its allowed distributional error. This should produce narrower models at comparable GP-like behavior, or permit the same parameter budget to be concentrated in the layers where finite-width…
Useful7/10
Difficulty5/10
Novelty7/10
✗ Mechanism failed
2026
Construct a sparse radius graph over input samples or learned node embeddings, compute its cycle space, and remove the subspace generated by sufficiently short cycles. Feed the remaining quotient-cycle coordinates or Betti-rank estimate to a graph neural network as a global topological feature, or use them to guide pooling so that local redundant loops are collapsed while global loops are retained. The paper predicts that the threshold L approximately equal to |log r| graph hops is the critical…
Useful7/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace ordinary dropout or soft sparsity penalties on a nonnegative spatial or token activation field with sublinear multiplicative stochastic dynamics. The field receives local diffusion or graph smoothing, while noise amplitude u^gamma vanishes at zero but is relatively strong near zero; this creates an absorbing zero state and may produce exact contiguous inactive regions. The module is suitable for feature maps, graph-node fields, token routing scores, or continuous neural operators.
Useful6/10
Difficulty6/10
Novelty8/10
Unverified
2026
Replace a dense token-mixing, MLP, or low-rank-adapter projection with a Bernoulli-signed sparse matrix normalized by the paper's predicted operator scale. Estimate the active representation dimension and use the effective scale to detect regimes in which extreme sparsity is likely to cause unstable amplification or dead rows.
Useful6/10
Difficulty4/10
Novelty5/10
Unverified
2026
Represent a block of candidate neural updates or adapter components by symmetric influence matrices and select one sign for each component so their aggregate spectral effect is small. This imports matrix discrepancy into low-rank adapters, expert aggregation, or structured quantization, where controlling the worst direction of interference may be more useful than minimizing entrywise error.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace a dense directional feature map with a finite stable-zonotope layer: a small set of large random vectors creates a hyperplane arrangement over the input sphere, while a cheaper small-jump residual gives continuous features inside each cell. The layer exposes an explicit compute-versus-resolution knob through the jump cutoff and can be used either as an embedding before an MLP or as a lightweight router for mixture-of-experts blocks.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Represent a higher-order neural computation as a bipartite incidence graph between node features and hyperedges, and assign each node-hyperedge incidence an anchor probability or learned anchor score. Add a regularizer that maximizes the predicted size of the surviving (k,n)-core under random node, hyperedge, or token dropout, thereby preventing structured pruning or routing from disconnecting essential higher-order computations. At inference, retain only incidences belonging to the predicted…
Useful6/10
Difficulty5/10
Novelty8/10
Unverified
2026
Replace fixed-budget token or patch pruning with greedy selection that combines a teacher-derived relevance score and Gaussian-process mutual information. Select an item when it is both relevant and non-redundant, and stop when the largest remaining information gain falls below a calibrated threshold instead of retaining a fixed number of items.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Attach a local BCM-trained binary adapter to a pretrained or frozen encoder, allowing new graph nodes or streaming examples to acquire representations without backpropagating through the main network. The adapter learns only from positive co-occurrence statistics and maintains sparse codes, providing a low-memory continual-learning path.
Useful6/10
Difficulty4/10
Novelty7/10
Unverified
2026
Replace ordinary stride-2 pooling by a stochastic block-to-center map that is equivariant under global sign reversal and monotone in every input spin. For a binary feature channel, the layer computes the probability of a positive coarse feature from the number of positive fine features, samples or relaxes the resulting Bernoulli variable, and learns only a constrained scalar rather than an unconstrained pooling kernel. The same construction can be applied independently to channels or to graph…
Useful6/10
Difficulty4/10
Novelty6/10
Unverified
2026
Construct a sparse fixed orthogonal mixer by repeatedly applying pi/4 rotations to randomly matched pairs of feature coordinates. Place this mixer before top-k feature pruning, sparse projection, or activation quantization so that information is spread across coordinates without using a dense random matrix.
Useful6/10
Difficulty4/10
Novelty6/10
Unverified
2026
Initialize a stable diagonal state-space layer with decay rates \(\omega_i=|\xi_i|\), where \(\xi_i\sim\mathcal N(\mu,\sigma^2)\), instead of using a narrowly clustered rate distribution. The nonzero density of rates near zero creates a population of slow modes whose aggregate impulse response has an algebraic tail, enabling long-horizon memory while every finite-dimensional mode remains exponentially stable.
Useful6/10
Difficulty4/10
Novelty6/10
Unverified
2026
Constrain a graph or token attention layer to behave like a row-stochastic diffusion with walk dimension \(\beta\), and choose its neighborhood radius from the paper's first-hitting-time tail bound. At propagation time \(t\), interactions with effective distance larger than \(r(t,\varepsilon)\) are removed because their total reachability probability is bounded by a prescribed error budget \(\varepsilon\).
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace or augment a dense attention similarity matrix with a Min-cone matrix generated by a monotone scalar sequence. The resulting matrix is positive semidefinite by construction, has only O(n) learned scalar parameters, and can be multiplied by values in O(n d) time using cumulative sums rather than forming an n-by-n matrix.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Use the number of tokens participating in an attention operation to adapt the projected query/key dimension according to the sharp Johnson–Lindenstrauss bound. Short attention windows receive a dimension limited by n-1, while longer windows use a logarithmic dimension, avoiding a fixed large head dimension when the current attention set contains too few points to require it.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Place doubly stochastic stream mixing immediately before a quantizer, activation compressor, or latent bottleneck and jointly optimize task loss with estimated code length. The paper's entropy argument says that this linear mixing cannot increase differential entropy, so it can provide cross-stream representation capacity without an ideal entropy-rate penalty; the entropy bottleneck then learns which feature values deserve bits.
Useful6/10
Difficulty4/10
Novelty4/10
Unverified
2026
Add a bounded routing state to an RNN, state-space model, or mixture-of-experts layer, with several neutral fixed points representing persistent modes. The state moves between modes when far from a fixed point but escapes each mode only polynomially when close to it, creating controllable long memory without setting a linear eigenvalue arbitrarily close to one. A temperature parameter selects between an entropy-rich phase using many modes and a low-entropy phase concentrated near one preferred…
Useful6/10
Difficulty5/10
Novelty8/10
Unverified
2026
Replace random or k-means initialization of a k-expert router with a moment-based range finder on a calibration batch of hidden states. Estimate a low-dimensional second-moment subspace, enlarge it using one-free-index third-Hermite contractions, and fit the router's expert centroids and weights only in this resulting subspace. The router can then operate on projected hidden states while retaining an optional small residual adapter.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace isotropic variance control in a bottleneck or router with a spectral polarization penalty that drives each latent direction toward either variance 0 or variance 1. The intended result is an automatically selected active subspace: inactive coordinates can be pruned or quantized aggressively, while active coordinates retain information instead of being uniformly attenuated.
Useful6/10
Difficulty4/10
Novelty7/10