Unverified
2026
Replace a collection of overlapping sliding-window features with approximately orthogonal incremental features: the length-m feature contains information not predictable from shorter consecutive windows. Use the paper's transition-operator Toeplitz precision matrix to decorrelate the resulting sequence of window features before attention, suppressing duplicated local evidence and improving conditioning.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Augment a stable diffusive state-space model with pair states formed from products of slow latent modes. Single modes represent ordinary long-wavelength diffusion, while pair modes represent the interacting hydrodynamic operators responsible for late-time tails in quartic observables. Use the pair states only for selected readout channels or a low-rank subset of mode pairs, preserving near-linear inference cost.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace an unconstrained recurrent state update by a locally parameterized invariant manifold h equals K of z, where the latent dynamics z at the next step equal R of z and preserve slow modes near a degenerate fixed point. Train the embedding and reduced map jointly with an invariance residual, while a weighted lattice norm discourages perturbations in distant channels or spatial sites from growing.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Instrument selected neural-network operators with cheap stochastic perturbations and estimate how much their outputs change under finite-precision perturbations. Use the resulting per-operator score to identify unstable kernels and selectively switch them to FP32, compensated accumulation, or a stable reformulation instead of running the entire model at high precision.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Learn a low-dimensional active frame for a neural scalar quantity on a curved latent manifold, rather than averaging gradients in unrelated ambient tangent spaces. Use the frame as the only input to a low-rank adapter or as a constraint on fine-tuning updates, with parallel transport making gradient statistics comparable across samples.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Construct a graph on tokens using a cheap similarity or k-nearest-neighbor rule, partition it into small-radius pieces, and use one pooled state per piece for cross-piece communication. The paper's bound guarantees that the total number of vertices exposed by all one-hop piece neighborhoods is at most 4n, so the routing graph can have linear rather than quadratic communication budget. The approximation is controlled by increasing the number of within-piece basis summaries rather than making…
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Attach a small dynamical observer to a neural ODE, RNN, or state-space model and make it estimate only a task-relevant functional of the hidden state, such as logits, value features, or control-relevant projections. Use an incremental quadratic constraint and a bounded-real penalty to make the observer robust to hidden-state nonlinearities and input disturbances, instead of reconstructing the full latent state.
Useful6/10
Difficulty6/10
Novelty5/10
Unverified
2026
Replace a dense order-p tensorized neural layer with a sparse random hypergraph tensor whose support probability is alpha = c/N^{p-1}. Assign independent zero-mean marks to active hyperedges, so every tensor moment scales as N^{-(p-1)} while the total variance entering each output remains controlled. The resulting locally tree-like computation graph should reduce parameters and multiply-adds without immediately destroying signal propagation.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Select the coordinates of a sparse adapter or sparse fine-tuning mask using both Fisher width and inverse-Fisher width. The mask should avoid parameter subsets that are cheap in the Fisher geometry but extremely large in the inverse-Fisher geometry, or vice versa, thereby controlling both prediction sensitivity and estimator-like uncertainty.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace ordinary sign-gradient or fixed-range gradient quantization in distributed training with a non-adaptive one-bit refinement block. Workers generate all localization and multiscale residue bits using public randomness before the server knows the current aggregate gradient; after decoding a coarse center, the server chooses which stored phase and scale interpretation to use. This should reduce clipping and variance for heavy-tailed gradients without requiring a second communication round.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Use the paper's effective-rank and eigengap-dependent covariance estimation rates to construct a confidence-aware low-rank bottleneck for transformer activations or key/value tensors. The bottleneck is enabled only when the top-p empirical eigenspace can be estimated more accurately than the desired compression error; otherwise the layer remains full-rank.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Construct sparse attention masks as regular or almost-regular hypergraphs: every query selects exactly w keys, while every key receives nearly equal incidence within each score or locality class. This preserves a fixed sparsity budget while removing systematic positional and head-level load imbalance that can cause hot keys, uneven memory traffic, and unstable gradients.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace a recurrent sequence classifier's unconstrained hidden-state alarm head with an online truncated-signature state and a first-hitting-time linear detector. The module summarizes local order information and cross-channel interactions while preserving exact compositional updates, making it suitable for long streaming sequences and early-exit decisions.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Turn the paper's graphon refinement operation into an adaptive mixture-of-experts or relational-attention router whose number of blocks is learned rather than fixed. Split a block when its members produce incompatible constraint gradients, and merge blocks when their relation rows are statistically indistinguishable, yielding a compact architecture that grows only where the data require extra structure.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace a large stable linear state-space or recurrent layer by a lower-order balanced realization computed from frequency-targeted controllability and observability Gramians. Use generalized low-rank ADI with imaginary-axis shifts concentrated at frequencies that dominate the training data, then retain states associated with the largest approximate Hankel singular values. This should reduce recurrent inference cost while preserving the layer's input-output response in the selected frequency…
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Build a low-dimensional basis for the principal module generated by the all-ones vector, then perform adjacency and degree propagation in compressed coordinates. On forests, the paper states that this module equals the automorphism-orbit module, so the reduced representation can preserve principal adjacency-degree information while using far fewer coordinates than the number of vertices.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Turn stochastic edge thinning into a mass-aware graph sparsifier: first form retained connected blobs, then connect blob summaries using the rank-one shortcut probability from the paper. Choose the retention rate below the explicit subcritical threshold so the contracted graph remains a collection of small decorations around leading blobs, reducing message-passing cost without creating an artificial giant component.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Add a nonlocal regularizer to nonnegative CNN feature maps that rewards activation-zero pixels lying deep inside a contiguous zero region. Unlike L1 sparsity, this penalizes isolated holes and favors block-like inactive areas that are more amenable to spatial skipping, channel gating, or structured pruning.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Use the paper's correspondence between KAN splines and finite-element or isogeometric shape functions to build coordinate-separable tensor-product trial layers. Replace additive coordinate aggregation with a multiplicative contraction of one-dimensional spline expansions, yielding an explicit tensor-product basis without storing a dense multidimensional grid.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace fixed sequence-to-sequence attention with a dynamically maintained tree of connected token groups. Groups can be fused to reduce the number of attention units or split when their representation is heterogeneous, while hypergraph connectivity and nestedness ensure that every intermediate hierarchy remains valid.
Useful6/10
Difficulty6/10
Novelty5/10
Unverified
2026
Replace a dense tensor-product positional encoding or first MLP layer with a hierarchical sparse-grid B-spline feature map. The network evaluates only localized basis functions indexed by multi-levels with bounded total level, reducing feature count while retaining high-order approximation for functions with mixed derivative regularity. The basis can initially be fixed and later fine-tuned jointly with the downstream network.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Compress a dense neural-network weight tensor with mode-wise selected fibers and a small sampled core, then convert the result into a Tucker layer for efficient execution. Select indices greedily using reconstruction residuals rather than random sampling, and fine-tune the compressed model to recover task accuracy.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Replace a dense block of a large neural-network weight matrix with an adaptively constructed cross approximation, but prevent premature termination using residual checks on O(m+n) additional diagonal-like entries. Accept a rank only after the probe residual has remained below tolerance for several consecutive iterations; otherwise continue adding pivots.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace a $K$-class softmax with $N$ binary hyperplane heads, where each class is represented by the signs of its projections onto fixed random directions. Train the embedding to reproduce these codewords and decode by nearest Hamming codeword. The paper's guarantee suggests that $N\approx 2\log_2 K+\log_2(1/\delta)$ can separate all class centers with high probability in sufficiently high dimension, giving a concrete width rule rather than choosing the number of binary heads heuristically.
Useful6/10
Difficulty4/10
Novelty5/10