△ Mechanism confirmed, baseline not beaten
2026
Replace independent architecture generation with a diffusion mutation kernel that starts from a known valid neural architecture, re-noises it for only a fraction of the diffusion horizon, and denoises it conditionally toward a new architecture. The resulting candidates should remain closer to the parent and retain validity at low mutation strength, while larger re-noising fractions should produce greater novelty and access to distinct architectural basins.
Useful8/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Construct a residual network from independently attachable modules, but permit only a number of modules whose aggregate feedback gain lies inside a delay-dependent admissible interval. Estimate deployed end-to-end latency and each module's local Jacobian gain, then reject or bypass additional modules when the predicted delayed-loop stability boundary is crossed. This turns variable-width or depth scaling into a falsifiable control problem rather than an empirical choice.
Useful8/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace black-box differentiation through an embedded LP decision with an analytic Jacobian computed from the LP’s active basis. A neural policy emits LP coefficients or right-hand sides; the LP returns the decision, while the backward pass uses the basis inverse and dual sensitivity, avoiding solver unrolling and finite-difference noise.
Useful8/10
Difficulty5/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Turn an iterative optimization or equilibrium computation inside a neural network into a differentiable layer whose backward pass solves the implicit adjoint system with conjugate gradients or GMRES using only automatic-differentiation matrix-vector products. This avoids storing unrolled iterations and avoids explicit Hessian or Jacobian construction, enabling longer solver horizons and lower-memory implicit architectures.
Useful8/10
Difficulty6/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Replace hard clipping or post-hoc asymmetric saturation with a dynamic output state that remains inside a prescribed asymmetric interval. A neural network emits a command uc, while the realized output u evolves through the APIR vector field, producing bounded actions, temporal smoothing, and gradients that remain available in the interior.
Useful8/10
Difficulty4/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Use a bounded function of the response to form a supervised, label-weighted covariance of the input and initialize the first neural layer from its leading outlier eigenspace. For vector-valued responses, use a matrix-valued response preprocessing map so several label statistics are combined in one lifted spectral estimator.
Useful8/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a standard ReLU surrogate with an input convex neural network whose hidden-to-hidden weights are constrained to be nonnegative. The network remains piecewise linear and expressive, but its convexity allows downstream minimization to use continuous ReLU epigraph constraints instead of binary activation variables, potentially eliminating the integrality bottleneck of neural optimization.
Useful8/10
Difficulty5/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Replace the ordinary gradient step by an update preconditioned by parameter directions actually excited by the observed part of the input. In a neural network, approximate this geometry with a masked Jacobian Gramian and damp directions with low observability, preventing arbitrary drift of parameters associated with missing features.
Useful8/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Compute an inner approximation of the states from which a neural controller can keep the plant inside a prescribed safe domain indefinitely, then use the resulting regulation map as a safety shield around the network. At each state, the network proposes an action, but the shield projects or replaces it with an action certified to remain in the invariant set.
Useful8/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a square dense projection in a Transformer or MLP with a trainable recursive butterfly matrix. The layer preserves multiscale channel interactions while constraining every complementary row-column block to rank at most k, reducing parameters and enabling recursive structured matrix-vector products. Unlike an arbitrary sparse layer, the construction has an explicit recursive factorization and a quasi-optimal approximation guarantee among matrices with the same butterfly rank.
Useful8/10
Difficulty6/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a dense neural-network weight tensor with a graph tensor network whose physical modes and internal edge ranks are specified by a sparse rank-adjacency matrix. Unlike tensor-train or hierarchical Tucker layers, the graph can contain selected cycles and skip connections between tensor modes, allowing the factorization topology to match correlations in the weight tensor. Fit the layer with GTN-SVD at a prescribed tolerance and compare accuracy, parameter count, and tensor-contraction…
Useful8/10
Difficulty6/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Attach several neural vector fields to a latent representation and train them to form a closed Lie algebra rather than learning unrelated augmentation directions. The resulting generators provide data-driven continuous transformations that can be used as equivariance constraints, while bracket closure and basis-rank penalties prevent degenerate or redundant generators.
Useful8/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Wrap a neural policy with a safety filter that minimally modifies its action so that a control-barrier inequality remains satisfied under bounded model mismatch and actuator saturation. Estimate mismatch between a learned plant or reference model and observed transitions online, then enlarge a conservative error margin and shrink the admissible safe set before solving the filter. The neural policy is unchanged when its action is safe, but receives a principled correction near state or action…
Useful8/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Before quantizing a matrix product, reparameterize its factors as A'=AT and B'=T^{-1}B, preserving the exact full-precision product while changing the quantization difficulty of each factor. Choose a positive diagonal T=diag(t_1,...,t_K) that minimizes predicted post-quantization product error, rather than using output-channel scaling or a fixed heuristic grid. The gauge can be shared across several products when transformed-copy cost matters.
Useful8/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Train a neural trial function for an elliptic PDE using a very-weak residual in which all derivatives act on fixed smooth test functions rather than on the neural network. This eliminates second-order reverse-mode or forward-mode automatic differentiation and allows low-regularity activations while retaining a least-squares objective over many test functions.
Useful8/10
Difficulty4/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace uniform collocation for a fixed random-feature neural PDE solver with sampling from the leverage-score density of the operator-applied features. Whiten the retained residual feature space before solving for output coefficients, so the sampled least-squares matrix has an identity-like expected Gram rather than inheriting severe anisotropy from the differential operator. The same construction can be used for a linearized neural network by treating Jacobian features as the trial functions.
Useful8/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace Euclidean or entrywise Kronecker fitting of a layer curvature matrix with its affine-invariant projection onto G = A tensor B. Use the resulting factors as a compact SPD preconditioner in the optimizer, while solving the projection through logarithmic residual partial traces and Armijo line search.
Useful8/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a deep feed-forward block by the fixed point z=phi(Wz+Vx+b), with the recurrent weight W constrained so that the fixed point is unique for every input. The same condition makes forward fixed-point iteration stable and makes implicit differentiation well-conditioned, allowing depth-independent memory usage while providing a measurable spectral failure boundary.
Useful8/10
Difficulty5/10
Novelty4/10
△ Mechanism confirmed, baseline not beaten
2026
Replace the direct Newton solve used in an implicit or equilibrium neural layer with a pseudo-arclength homotopy solve that augments the potentially singular layer Jacobian by one continuation direction. The layer can then track a solution branch through generic folds, where ordinary inversion becomes unbounded, while selecting the minimum-norm state and continuation update.
Useful8/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Build a residual module whose state explicitly contains both a persistent context representation and an accumulator. Each residual branch computes one learned correction and adds it to the accumulator, instead of forcing every layer to represent the complete output from scratch. This provides a concrete solver-like architecture for high-dimensional regression and iterative latent prediction.
Useful7/10
Difficulty4/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Construct a continuous-depth or recurrent residual block with a prescribed polynomial Lyapunov decay near its equilibrium. The architecture combines a fixed radial stabilizer with a learned component that is constrained to have zero radial projection, allowing slow algebraic transients and long memory while preventing asymptotic hidden-state growth.
Useful7/10
Difficulty6/10
Novelty8/10
△ Mechanism confirmed, baseline not beaten
2026
Replace the usual isotropic Gaussian random Fourier features with a frequency distribution matched to the expected spectral regularity of the target function. For coordinate fields, operator-learning maps, or PDE solution surrogates, this should place more features where the target Fourier energy lies and improve approximation at the same feature count. Stabilize the resulting feature matrix with whitening or ridge regression because spectral accuracy can create severe ill-conditioning.
Useful7/10
Difficulty4/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Represent a large linear map acting on a Cartesian 3D grid and multiple physical channels as a TT-matrix, while retaining separate TT blocks for channel couplings that have different semantics. Apply the layer by sequential contractions with TT cores rather than materializing a dense matrix or a full 3D convolution kernel. Rank truncation provides an explicit accuracy-versus-memory knob and can be applied after optimizer updates.
Useful7/10
Difficulty5/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Replace an unconstrained neural ODE or recurrent update field with the negative gradient of a learned scalar energy \(E_\theta(z,t)\). The resulting hidden-state dynamics have an exact Lyapunov certificate: energy decreases continuously, bounded trajectories cannot exhibit nonstationary recurrence, and the Łojasiewicz mechanism predicts convergence to a single equilibrium rather than persistent oscillation or chaos.
Useful7/10
Difficulty5/10
Novelty6/10