✗ Failed on benchmark
2026
Prune hidden units only after testing whether their nonlinear gate is task-visible and downstream-used. For ReLU, a unit is removable when its preactivation does not cross zero on the task patch or its outgoing weight column is zero; this is a more structural criterion than weight magnitude and can be applied during width search or post-training compression.
Useful6/10
Difficulty3/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Represent a large positive semidefinite neural operator as the sum of two Kronecker products and regularize an efficiently computed upper bound on its largest eigenvalues. The bound controls not only the spectral norm but every top-k eigenvalue sum, allowing a tunable penalty on concentrated or unstable directions without constructing the exponentially larger operator.
Useful6/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Replace nominal hidden-unit count by the number of distinct realized ReLU kink hyperplanes, then regularize or prune this effective count. Neurons whose normalized affine boundaries coincide can be exactly merged by summing their canonical coefficients, reducing memory without changing the represented function and aligning the model's complexity measure with the theorem.
Useful6/10
Difficulty5/10
Novelty6/10
✗ Mechanism failed
2026
Use the paper's lower bound as a feasibility test for robust interpolation: if a model is asked to fit below the estimated noise floor while maintaining a small Lipschitz constant, automatically increase effective width or relax the fit target. This prevents optimization from wasting compute on an impossible low-sensitivity solution and provides a principled width schedule for noisy regression or classification.
Useful6/10
Difficulty4/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Separate a neural network into nonlinear hidden parameters and a linear output layer. Solve the output layer exactly by least squares, then update hidden parameters with a truncated-pseudoinverse Gauss-Newton step that discards numerically singular directions.
Useful6/10
Difficulty6/10
Novelty5/10
✗ Mechanism failed
2026
Replace an unpreconditioned conjugate-gradient solve for a damped Gauss–Newton step with a two-level algebraic preconditioner derived from local Jacobian-row supports. Use overlapping local parameter blocks as Schwarz subdomains and a coarse basis containing low-energy local modes, so the optimizer can correct both localized and globally coupled parameter errors.
Useful6/10
Difficulty6/10
Novelty6/10
✗ Failed on benchmark
2026
Replace a fixed soft-threshold, ReLU-like gate, or manually chosen activation shrinkage with a monotone learned shrinkage function fitted by an observed-data quadratic-risk criterion. The gate can interpolate between identity, ridge-like attenuation, hard thresholding, and lasso-like soft thresholding, allowing each layer or channel group to adapt its bias–variance tradeoff from the current minibatch.
Useful6/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a binary classifier's unconstrained final logit with a differentiable likelihood-ratio head based on two squared Mahalanobis radii in a learned embedding space. Approximate the shared radial generator with a small fractional-power basis, allowing the head to model heavy-tailed class geometry that an affine QDA logit cannot represent while remaining much smaller than a generic nonlinear head.
Useful6/10
Difficulty5/10
Novelty6/10
✗ Mechanism failed
2026
Replace a dense weight matrix by a cross approximation built from selected rows and columns rather than by a conventional truncated SVD. Periodically refresh the selected indices using residual leverage scores, warm-starting from the previous factorization so that the compressed layer can track weight changes during fine-tuning.
Useful6/10
Difficulty5/10
Novelty5/10
✗ Mechanism failed
2026
Replace the first dense layer on q-ary categorical features by a Fourier interaction layer containing only monomials whose coordinate support is at most s. Use a Bohnenblust–Hille-inspired quasi-norm on coefficients, separately for each interaction order, to prevent a small number of high-order interactions from dominating the output. The resulting model has an explicit interaction-order knob and can be tested against a dense MLP at matched parameter count.
Useful6/10
Difficulty5/10
Novelty7/10
✗ Mechanism failed
2026
Replace ordinary hidden-weight decay with a recursive ℓ1 variation penalty on the coefficients used to combine activated functions from the previous layer. Use normalized activations \(\sigma_s(t)=\sigma(st)/s\) so that the learned scale parameter \(s\) controls feature shape separately from the coefficient magnitude charged by the variation norm.
Useful6/10
Difficulty5/10
Novelty5/10
✗ Mechanism failed
2026
Replace an unconstrained scalar activation or nonnegative gate with a finite positive mixture of rational Bernstein basis functions. The learned function is monotone and concave on the nonnegative half-line, while its derivatives have controlled alternating signs; this can prevent pathological feature amplification and gives an interpretable shape prior. Use the paper's sharp exponent restriction τ≤1/2 rather than treating the power as an arbitrary hyperparameter.
Useful6/10
Difficulty4/10
Novelty7/10
✗ Mechanism failed
2026
Represent every mesh interface degree of freedom by one feature copy per incident cell, and apply local neural blocks directly to these cell tensors. Enforce inter-cell consistency with valence-weighted averaging only after selected layers or hierarchy transitions, avoiding repeated construction of a global sparse graph or assembled feature vector. This is suited to adaptive quadtrees, octrees, and finite-element neural operators.
Useful6/10
Difficulty5/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Build an input-conditioned unitary transformation as an ordered product of exponentials of anti-Hermitian matrices, with each factor controlled by a univariate function of one input coordinate or one learned scalar projection. This replaces a dense multivariate matrix-valued controller with separable scalar nonlinearities while preserving exact unitarity at every forward pass.
Useful6/10
Difficulty6/10
Novelty7/10
✓✓ Beats tuned baseline
2026
For a fractional Dirichlet problem, replace a free coordinate network N_theta(x) with u_theta(x)=d(x)^a N_theta(x), where d(x)=dist(x,boundary) and 0<a<1 is the fractional order. Train the regular quotient v_theta=u_theta/d^a=N_theta and use a weighted gradient loss that reflects the paper's boundary estimate.
Useful6/10
Difficulty4/10
Novelty7/10
✗ Mechanism failed
2026
Replace a dense channel or token-mixing matrix with a product of positive bidiagonal factors, so information propagates through a controlled sequence of local couplings rather than arbitrary signed interactions. Initialize the factors from the paper's barycentric-subdivision factorization, then learn positive diagonal and off-diagonal parameters; the resulting map is structured, parameter-efficient, and constrained to remain totally positive.
Useful6/10
Difficulty5/10
Novelty8/10
✓✓ Beats tuned baseline
2026
Train an unconstrained branch and a geometry-aware branch in parallel, then learn how much to trust the analytic branch. This preserves the benefit of explicit geometry on correctly specified tasks while allowing the model to ignore a misleading or irrelevant prior.
Useful6/10
Difficulty3/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Add a conservative correction after low-rank tensor compression so selected linear moments of an activation or learned state are exactly preserved. This can reduce tensor rank and memory without allowing compression error to accumulate in physically meaningful global quantities.
Useful6/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a generic neural constitutive law or energy model with an ICNN that consumes the positive singular values of a deformation-like matrix and is convex and coordinatewise nondecreasing in those inputs. Train it as a lower approximation to a nonconvex target energy, so the network acts as a computationally cheap sufficient polyconvex-envelope surrogate rather than merely interpolating unstable samples.
Useful6/10
Difficulty4/10
Novelty4/10
✓✓ Beats tuned baseline
2026
Represent the PINN solution in a restricted polynomial or Taylor basis whose exponent set is supplied by tropical support analysis, instead of asking an MLP to discover the local series structure from scratch. The restriction removes coefficients that cannot occur in the formal solution, reducing trainable degrees of freedom and preventing spurious low-order or singular terms.
Useful6/10
Difficulty5/10
Novelty8/10
✓✓ Beats tuned baseline
2026
Replace an unconstrained feature vector entering a rational or resolvent-like neural operator by a polynomial feature whose first nonzero Taylor coefficient lies in a pole-safe subspace. For a pole of order m, the simplest guaranteed construction is psi(z)=(z-beta)^m v, which makes Q(z)psi(z) bounded even when Q(z) diverges. For lower-order cancellation, solve linear constraints among Taylor coefficients of psi so that all negative Laurent powers vanish.
Useful6/10
Difficulty5/10
Novelty8/10
✗ Mechanism failed
2026
Replace fixed-strength projection or constraint-repair steps during low-rank neural fine-tuning with a regularized affine subproblem whose damping is proportional to the current distance from the model manifold. Use strong damping when a gradient update leaves the low-rank manifold substantially, then automatically remove the damping near a clean intersection so that the method can recover higher-order local convergence. This is suitable for LoRA-style updates, structured matrix compression…
Useful6/10
Difficulty6/10
Novelty6/10
✗ Mechanism failed
2026
Bootstrap the optimizer curvature scale from a deliberately nondegenerate pair of gradient queries, then perform steepest descent in lp geometry with a local secant backtracking rule. The method does not require a supplied learning rate, smoothness constant L, initial distance R, or optimum value f*, and it automatically uses the dual norm associated with p.
Useful6/10
Difficulty5/10
Novelty6/10
✗ Mechanism failed
2026
Replace scalar neural activations by pairs of nonnegative channels whose ratio represents the signed or unsigned activation. Implement multiplication and addition through pair algebra, and renormalize each pair because the representation is invariant under multiplying both rails by the same positive scalar. This creates an explicitly bounded, cancellation-aware arithmetic layer for deep multiplicative MLPs, rational networks, and neural fields.
Useful6/10
Difficulty5/10
Novelty7/10