✗ Failed on benchmark
2026
Replace selected residual, recurrent, or state-space blocks by modules whose input-output Jacobians satisfy an IODP inequality throughout a prescribed activation domain. The constraint controls incremental amplification between two trajectories without requiring either trajectory to remain near one fixed equilibrium, so it should improve robustness to changing contexts and prevent exploding long-horizon sensitivities.
Useful8/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace the ordinary gradient step by an update preconditioned by parameter directions actually excited by the observed part of the input. In a neural network, approximate this geometry with a masked Jacobian Gramian and damp directions with low observability, preventing arbitrary drift of parameters associated with missing features.
Useful8/10
Difficulty6/10
Novelty6/10
✗ Mechanism failed
2026
For z neural branches that share a target, state, or routing observation, add a penalty on fluctuations in the branch direction visible to that shared signal. This implements the paper's centered-square conditioning mechanism: branches remain locally independent in hidden directions, while collective deviations that would produce inconsistent shared outputs are suppressed.
Useful8/10
Difficulty4/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Represent each neural module as a Hamiltonian storage system and connect modules through a state-dependent skew or Dirac interconnection instead of arbitrary residual additions. The coupling may change with the hidden state, but its internal power contribution cancels exactly, so total stored energy is controlled only by external inputs and explicitly added dissipation.
Useful8/10
Difficulty5/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Attach several neural vector fields to a latent representation and train them to form a closed Lie algebra rather than learning unrelated augmentation directions. The resulting generators provide data-driven continuous transformations that can be used as equivariance constraints, while bracket closure and basis-rank penalties prevent degenerate or redundant generators.
Useful8/10
Difficulty6/10
Novelty7/10
✗ Mechanism failed
2026
Replace direct learning of a highly cancelling signed observable with a quotient-space model over symmetry orbits of inputs. Predict a physically constrained baseline for each family and use an LSTM or set/graph encoder only for the residual many-body correlation, then aggregate family predictions with known signed weights instead of forming a noisy sample-level ratio.
Useful8/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
For a neural network with a trainable linear head or low-rank adapter, store feature vectors from recent minibatches and select a finite set that is sufficiently independent. Apply Modified Gram-Schmidt to obtain orthonormalized memory directions, then add residual corrections along these directions so the local parameter-error dynamics have an identity coefficient matrix rather than a poorly conditioned empirical Gramian. The method predicts a sharp transition after the buffer first contains…
Useful7/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Replace a single smooth neural vector field with a finite collection of smooth subnetworks selected by learned affine hyperplanes. The architecture exposes switching geometry directly, allowing it to represent friction-like or threshold dynamics without approximating discontinuities using excessively steep activations.
Useful7/10
Difficulty5/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Replace the final layer of a neural predictor with Bayesian linear regression over deterministic trigonometric features, retaining a computable posterior variance and a high-probability confidence envelope over the full bounded input domain. Use this envelope to reject unsafe actions, downweight uncertain training targets, or restrict optimizer updates in regions where the network is extrapolating.
Useful7/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a single residual stream or unconstrained hyper-connection with S parallel feature streams whose cross-stream mixing matrix is doubly stochastic. Parameterize the matrix with Sinkhorn normalization so every layer preserves total stream mass while still learning adaptive information routing. This is a low-overhead alternative to dense cross-stream attention and should reduce stream explosion, collapse, and sensitivity to depth.
Useful7/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace the exact matrix-polar normalization in Muon with the smoothed feedback \(h_\epsilon(M)=M(M^\top M+\epsilon I)^{-1/2}\). This retains singular-vector-aware updates and approximately unit-normalizes dominant spectral modes, but avoids unstable behavior when the momentum matrix is rank deficient or has tiny singular values.
Useful7/10
Difficulty5/10
Novelty4/10
✓✓ Beats tuned baseline
2026
Construct hidden dynamics from permutation-equivariant vector fields and impose antisymmetry through an explicit antisymmetrizing readout. This prevents optimization from learning multiple equivalent copies of the same configuration and makes forbidden symmetry violations exactly zero, rather than merely penalizing them. The design applies to set models, particle systems, graph networks, and architectures handling unordered tokens.
Useful7/10
Difficulty5/10
Novelty4/10
△ Mechanism confirmed, baseline not beaten
2026
Use the switched nonlinear extension to distinguish stability of the linearized modes from stability of the full neural dynamics. Stabilize worst-case linear products and limit the variation of each nonlinear Jacobian inside a specified radius, yielding an explicit local basin estimate and a penalty that prevents mode interactions from destroying attraction.
Useful7/10
Difficulty5/10
Novelty8/10
△ Mechanism confirmed, baseline not beaten
2026
Insert a small continuous-time Markov latent module between a neural encoder and decoder, with input-dependent transition rates and a fixed library of graph topologies such as directed cycles, reversible chains, and branching motifs. The output is an observable of the stationary distribution, while a learned convex mixture over topology-specific response curves constrains the network to represent responses as combinations of interpretable nonequilibrium mechanisms.
Useful7/10
Difficulty6/10
Novelty8/10
✓✓ Beats tuned baseline
2026
Replace isotropic input-Jacobian regularization with a positive semidefinite, input-dependent metric learned jointly with the network. The metric uses diagonal scaling to suppress sensitivity in nuisance directions and a structured orthogonal rotation to discover combinations of input coordinates in which smoothness is task-useful.
Useful7/10
Difficulty6/10
Novelty6/10
Unverified
2026
Insert a fixed expander channel before a covariance-dependent feature transformation. The channel repeatedly conjugates the feature covariance by a constant number of sparse Pauli/CNOT unitaries, preserving total feature energy while contracting anisotropic covariance components. Use the mixed covariance for whitening or as a regularized normalization statistic, and test whether it gives more stable training than dense whitening or an explicit isotropy penalty.
Useful6/10
Difficulty5/10
Novelty8/10
Unverified
2026
Insert a projection-space mixer that combines several fixed or learned directions using reciprocal correlations with the current feature, then normalize the result. The exact construction has a universal beta law for its squared input-output cosine, so it can create controlled angular diversity while remaining deterministic and independent of the chosen direction dictionary.
Useful6/10
Difficulty5/10
Novelty8/10
Unverified
2026
Replace an unconstrained hidden-to-hidden interaction in an MLP or transformer feed-forward block by two gauge-related branches. Split channels with an orthogonal involution Θ, constrain the learned interaction K to anticommute with Θ, and use opposite signs of K in paired branches. This creates a testable inductive bias in which the learned interaction only transfers information between the two channel subspaces.
Useful6/10
Difficulty5/10
Novelty8/10
Unverified
2026
Construct a generative or recurrent neural architecture with several contractive or mildly expanding branches, and explicitly control the geometric complexity of its invariant set using the sub-additive singular-value pressure of branch-Jacobian products. Instead of regularizing only the operator norm, the model can preserve anisotropic directions while targeting a desired attractor dimension, potentially improving coverage of structured data without uncontrolled folding or collapse.
Useful6/10
Difficulty6/10
Novelty8/10
Unverified
2026
For a neural network predicting A coupled concentration or density fields, decompose Fourier-space fluctuations into a charge direction and its charge-neutral composition subspace. Hard-project the predicted fields to eliminate the global charge mode, and regularize their low-wavenumber covariance so that neutral modes retain finite susceptibility while the charge structure factor follows the Coulombic suppression S_ZZ(k) proportional to k squared. This should improve long-range physical…
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
For a neural block with matrix-valued activations and transformation Y = A X B, regularize the exact coupled spectrum of the two-sided map instead of penalizing A and B independently. A large singular direction in A is penalized more strongly when the corresponding singular direction in B is also large, directly controlling joint feature amplification.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Attach a finite mixture of zonotopes to each uncertain neural input or hidden state, and propagate every mixture component through affine layers and conservative nonlinear relaxations. When the number of components grows, merge components only with an enclosing zonotope and sum their probability masses, preserving a formal lower bound on the probability that the true activation lies in the represented set.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Augment a neural-network update with an auxiliary, damped stochastic branch that acts like the paper's floating dissipative reservoir. A trainable mixing phase \(\phi\) combines the task-gradient branch and auxiliary branch; \(\phi\) is adapted to make the auxiliary response to a chosen control perturbation nearly zero while retaining a finite task-gradient response. The intended benefit is selective insensitivity to nuisance hyperparameters or perturbations, with a measurable response peak…
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace an unconstrained multiplicative interaction between two nonnegative neural features by a lifted gate whose first and second moments satisfy the paper's semidefinite relaxation for the set F = {(x1,x2): x1,x2 >= 0, x1 x2 <= 1}. Insert the gate into an MLP, attention score, or MoE router to prevent explosive feature products while retaining a tractable convex feasible set.
Useful6/10
Difficulty7/10
Novelty8/10