△ Mechanism confirmed, baseline not beaten
2026
Parameterize a trainable weight update as \(\Delta W=UV^{\top}\) with an excessive initial rank \(r\), and penalize active columns using an exact column \(\ell_{2,0}\) penalty. Increase \(\lambda\) along a warm-started path and hard-delete redundant paired columns, producing an automatically selected rank without training a separate model for every candidate rank. Apply scale balancing after each update so pruning decisions are invariant to reciprocal rescaling of factor pairs.
Useful7/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a soft auxiliary load-balancing loss with a deterministic router correction that balances expert assignments separately within confidence or score shells. The correction preserves the number of tokens assigned to each shell while making every expert receive equal or nearly equal mass inside each shell, preventing high-confidence tokens from monopolizing a subset of experts.
Useful7/10
Difficulty5/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Replace independently parameterized scalar, vector, and higher-order neural outputs with consecutive spaces of ReLU-power differential forms linked by an exact exterior-derivative layer. The network can then produce curl-free, divergence-free, or more general closed fields by construction, while the complex prevents artificial null-space modes that commonly appear when differential constraints are enforced only through sampled residual losses.
Useful7/10
Difficulty5/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Build a policy or prediction head that scores atomic graph entities once and converts those scores into logits for legal composite actions through an instance-specific incidence matrix. The neural parameters are independent of the number of actions and their composition, so the same model can process graphs with different action counts and compatibility structures without padding every action space to a global maximum.
Useful7/10
Difficulty4/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a portion of quadratic key-value attention or an external episodic table with a per-sample matrix fast memory updated by rank-one delta corrections. The memory directly learns a linear key-to-value map and can be carried across sequence segments, providing cheap online adaptation with constant state size per head.
Useful7/10
Difficulty5/10
Novelty5/10
✗ Failed on benchmark
2026
Add a differentiable orientation-pooling layer after steerable filters or an orientation-bin expansion. It aggregates unoriented line evidence using doubled-angle vectors, so a feature at angle θ is identical to one at θ+π, while symmetric orientations cancel naturally instead of producing an arbitrary mean angle. Feed the network both the Cartesian order parameter and its magnitude-based confidence.
Useful7/10
Difficulty4/10
Novelty6/10
✗ Failed on benchmark
2026
Replace ordinary graph propagation by diffusion with a positive node-dependent mass matrix \(\mathbf V\), so high-volume nodes update slowly and low-volume nodes update rapidly. Use node volumes as fixed metadata, a function of degree, or learned positive gates; this makes the architecture sensitive to dynamical localization that degree-normalized GCNs cannot represent.
Useful7/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Build an SPD classifier and residual head directly from Cholesky factors, using lower-triangular differences and matrix-power terms instead of generic eigendecomposition-based logarithm operators. This retains covariance geometry while making positive-definiteness automatic and backpropagation more numerically stable for minibatch training.
Useful7/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
For coupled recurrent or state-space modules that represent oscillatory or periodic signals, explicitly account for communication or attention delay in the characteristic equation. Tune the coupling gain or add a phase-lead compensator so that the desired latent frequency remains a closed-loop mode instead of being shifted by small delays.
Useful7/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Construct overlapping attention windows from a block design instead of using one dense sequence-by-sequence attention matrix. Every token appears in exactly $r$ windows and every token pair co-occurs in exactly $\lambda$ windows, giving uniform coverage and avoiding the uneven connectivity of arbitrary sparse masks.
Useful7/10
Difficulty5/10
Novelty6/10
✗ Failed on benchmark
2026
Split hidden dynamics into relaxation bands when the Jacobian spectrum has a gap, evolve each band with its own timescale, and retain an explicit cross-band exchange term. This yields a principled dual-timescale RNN or SSM rather than choosing fast and slow branches heuristically.
Useful7/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Build an autoencoder whose decoder outputs a monotone quantile function rather than an unconstrained spatial field. The latent representation can be compressed with POD or a neural bottleneck in CDT space, while the decoder guarantees valid transport maps and therefore avoids negative densities, mass drift, and spurious oscillations common in unconstrained reduced-order neural decoders.
Useful7/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Use the recursive errors-in-variables subspace spectrum as a controller for the width of a latent SSM rather than fixing the state dimension in advance. Neurons or state channels are added when corrected covariance eigenvalues rise above the noise floor and pruned when they remain below it, producing a model-order-adaptive recurrent architecture for nonstationary streams.
Useful7/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Use small-gain diagnostics to jointly learn module normalization and a communication partition rather than imposing a fixed global spectral constraint. Clusters should be formed around high-gain feedback loops, because grouping weakly related modules cannot improve the certificate and only adds bookkeeping.
Useful7/10
Difficulty7/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Grow a neural network by appending a trainable block together with an analytically initialized inverse block, so the newly added depth is exactly the identity at insertion time. After insertion, untie and optimize the two blocks independently; this preserves the current function while providing additional trainable degrees of freedom. For architectures with one expensive mixing operation followed by cheap channelwise blocks, the same construction can increase depth without repeatedly paying for…
Useful7/10
Difficulty5/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Replace one globally shared LoRA adapter with a small set of adapters whose task membership is chosen by residual redundancy. Tasks with strongly correlated validation residuals share an adapter, while tasks with weak or antagonistic residual dependence receive separate adapters. Recompute the partition periodically so the architecture follows the coupling that remains after training rather than correlations in the raw labels or initial gradients.
Useful7/10
Difficulty5/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Replace raw molecular orientation vectors with local scalar features invariant under common three-dimensional rotations and the apolar transformation u_i -> -u_i. Feed these channels to a CNN autoencoder, VAE, or contrastive encoder so that configurations on the same physical symmetry orbit have identical inputs or latent codes. This should improve unsupervised phase discovery without supplying order-parameter labels.
Useful7/10
Difficulty4/10
Novelty5/10
✗ Failed on benchmark
2026
Construct a neural activation bottleneck by projecting hidden states into a fixed covariance-eigenbasis and retaining only the d largest-magnitude coordinates per sample. For Gaussian, decorrelated activations, the paper proves that adaptive top-d selection in the PCA basis has no greater expected residual energy than adaptive top-d selection after any other orthogonal rotation. This provides a principled alternative to learning an unrestricted rotation before sparsification.
Useful7/10
Difficulty3/10
Novelty5/10
✗ Failed on benchmark
2026
Replace an unconstrained recurrent block with two coupled modules: a contractive perceptual estimator and an input-to-state-stable cognitive state transition. Spectral normalization and a controlled Euler residual step enforce a quantitative gain condition, preventing hidden-state explosion while retaining long memory when the contraction factor is chosen close to one.
Useful7/10
Difficulty5/10
Novelty6/10
✗ Failed on benchmark
2026
Use the paper's stable periodic orbits and border-collision transitions as an intentional memory mechanism in a recurrent module. Different input-dependent parameter settings can place the same cell in fixed-point, period-2, or higher-period regimes, allowing a compact state to encode discrete modes without allocating one separate neural attractor per mode.
Useful7/10
Difficulty7/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a network head that independently predicts coupled physical source terms with a low-dimensional rate head followed by a fixed stoichiometric map. This makes conservation of total mass or other linear invariants exact by construction and leaves the network responsible only for learning the kinetics of admissible exchange channels.
Useful7/10
Difficulty3/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Build a neural dynamical block whose hidden state contains differential variables and Lagrange multipliers, with a singular descriptor matrix enforcing constraints during propagation. This avoids the drift and ill-conditioning that can arise when exact constraints are represented only by a penalty term.
Useful7/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace dense query-key attention with an adaptive cross approximation constructed from selected query and key pivot tokens. At each rank, choose the pivot pair that removes large estimated residual energy, update the residual by a rank-1 cross correction, and stop when the residual estimate reaches a target tolerance. The resulting factorization computes approximate attention using a small number of landmark interactions while adapting to the actual token distribution.
Useful7/10
Difficulty6/10
Novelty5/10
✗ Failed on benchmark
2026
Train a neural surrogate to predict outputs in a source-domain ZCA-whitened space, then adapt to a shifted domain using only the shifted domain's output mean and covariance. At inference, transport the network prediction through the target covariance square root, yielding a weight-free correction that preserves output-coordinate semantics and can be applied to MLP, CNN, graph-NN, or transformer regressors.
Useful7/10
Difficulty4/10
Novelty5/10