Unverified
2026
Construct a reusable ReLU trunk that emits approximate univariate powers or Legendre-polynomial features for each input coordinate, then combine them with a linear or low-rank polynomial head. This gives a compact explicit basis for smooth functions and can replace a large generic MLP in low-dimensional scientific regression or serve as a frozen or partially trainable front-end.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
For smooth coordinate-based regression, replace a width-heavy MLP with a deliberately narrow but deeper ReLU network and choose depth and width using the paper's analytic-function approximation law. The hypothesis is that, at fixed parameter count, increasing depth gives a larger reduction in approximation error than increasing width when the target is close to analytic.
Useful6/10
Difficulty4/10
Novelty7/10
Unverified
2026
Add a differentiable regularizer to neural networks that learn sparse Fourier coefficients or trainable Fourier-feature frequencies. It penalizes predicted energy just outside the training interval when that energy exceeds the theorem-shaped envelope relative to observed in-domain L2 energy, discouraging cancellation patterns that fit the observed interval but explode nearby.
Useful6/10
Difficulty3/10
Novelty8/10
Unverified
2026
Replace an unconstrained Fourier-feature block in an implicit neural representation or coordinate MLP with a sparsity-aware layer whose output gain is normalized according to the distance outside the training interval. The normalization uses the paper's endpoint law, preventing a small in-domain Fourier representation from producing arbitrarily large outputs just beyond the observed coordinate range.
Useful6/10
Difficulty4/10
Novelty7/10
Unverified
2026
Represent input or parameter uncertainty locally by a low-order polynomial expansion of the network output, and compute only task-relevant directional third- and fourth-order moments. Add a penalty that calibrates or controls projected skewness and kurtosis, allowing the model to represent bent or elongated confidence regions without constructing a full dense moment tensor.
Useful6/10
Difficulty5/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Replace the usual explicit residual update with a nonstandard general-linear block containing several internal feature stages. The effective step is a positive denominator function rather than the raw depth step, allowing the block to take large nominal steps while damping the update and preserving bounded activations. This is most promising for deep residual MLPs, neural ODE discretizations, and state-space sequence models where exploding hidden states limit usable depth.
Useful6/10
Difficulty6/10
Novelty6/10
✗ Mechanism failed
2026
Construct a differentiable approximate projection layer for a convex feasible set by averaging Gaussian perturbations conditioned on feasibility. Unlike hard projection or penalty methods, the output is smooth in the input and inherits firm nonexpansiveness in the convex case.
Useful6/10
Difficulty4/10
Novelty5/10
✗ Mechanism failed
2026
Use a teacher and student with comparable hierarchical layers, first matching their representations through a learned affine map and then explicitly encouraging one-to-one coordinate correspondence up to permutation and positive rescaling. This operationalizes the paper's weak-to-strong alignment mechanism for representation distillation, model merging, or stabilizing independently trained replicas.
Useful6/10
Difficulty5/10
Novelty6/10
✗ Failed on benchmark
2026
Prune hidden units only after testing whether their nonlinear gate is task-visible and downstream-used. For ReLU, a unit is removable when its preactivation does not cross zero on the task patch or its outgoing weight column is zero; this is a more structural criterion than weight magnitude and can be applied during width search or post-training compression.
Useful6/10
Difficulty3/10
Novelty5/10
✗ Mechanism failed
2026
Use the paper's lower bound as a feasibility test for robust interpolation: if a model is asked to fit below the estimated noise floor while maintaining a small Lipschitz constant, automatically increase effective width or relax the fit target. This prevents optimization from wasting compute on an impossible low-sensitivity solution and provides a principled width schedule for noisy regression or classification.
Useful6/10
Difficulty4/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Separate a neural network into nonlinear hidden parameters and a linear output layer. Solve the output layer exactly by least squares, then update hidden parameters with a truncated-pseudoinverse Gauss-Newton step that discards numerically singular directions.
Useful6/10
Difficulty6/10
Novelty5/10
✗ Failed on benchmark
2026
Replace a fixed soft-threshold, ReLU-like gate, or manually chosen activation shrinkage with a monotone learned shrinkage function fitted by an observed-data quadratic-risk criterion. The gate can interpolate between identity, ridge-like attenuation, hard thresholding, and lasso-like soft thresholding, allowing each layer or channel group to adapt its bias–variance tradeoff from the current minibatch.
Useful6/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a binary classifier's unconstrained final logit with a differentiable likelihood-ratio head based on two squared Mahalanobis radii in a learned embedding space. Approximate the shared radial generator with a small fractional-power basis, allowing the head to model heavy-tailed class geometry that an affine QDA logit cannot represent while remaining much smaller than a generic nonlinear head.
Useful6/10
Difficulty5/10
Novelty6/10
✗ Mechanism failed
2026
Replace a dense weight matrix by a cross approximation built from selected rows and columns rather than by a conventional truncated SVD. Periodically refresh the selected indices using residual leverage scores, warm-starting from the previous factorization so that the compressed layer can track weight changes during fine-tuning.
Useful6/10
Difficulty5/10
Novelty5/10
✗ Mechanism failed
2026
Replace the first dense layer on q-ary categorical features by a Fourier interaction layer containing only monomials whose coordinate support is at most s. Use a Bohnenblust–Hille-inspired quasi-norm on coefficients, separately for each interaction order, to prevent a small number of high-order interactions from dominating the output. The resulting model has an explicit interaction-order knob and can be tested against a dense MLP at matched parameter count.
Useful6/10
Difficulty5/10
Novelty7/10
✗ Mechanism failed
2026
Replace ordinary hidden-weight decay with a recursive ℓ1 variation penalty on the coefficients used to combine activated functions from the previous layer. Use normalized activations \(\sigma_s(t)=\sigma(st)/s\) so that the learned scale parameter \(s\) controls feature shape separately from the coefficient magnitude charged by the variation norm.
Useful6/10
Difficulty5/10
Novelty5/10
✗ Mechanism failed
2026
Replace an unconstrained scalar activation or nonnegative gate with a finite positive mixture of rational Bernstein basis functions. The learned function is monotone and concave on the nonnegative half-line, while its derivatives have controlled alternating signs; this can prevent pathological feature amplification and gives an interpretable shape prior. Use the paper's sharp exponent restriction τ≤1/2 rather than treating the power as an arbitrary hyperparameter.
Useful6/10
Difficulty4/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Build an input-conditioned unitary transformation as an ordered product of exponentials of anti-Hermitian matrices, with each factor controlled by a univariate function of one input coordinate or one learned scalar projection. This replaces a dense multivariate matrix-valued controller with separable scalar nonlinearities while preserving exact unitarity at every forward pass.
Useful6/10
Difficulty6/10
Novelty7/10
✓✓ Beats tuned baseline
2026
For a fractional Dirichlet problem, replace a free coordinate network N_theta(x) with u_theta(x)=d(x)^a N_theta(x), where d(x)=dist(x,boundary) and 0<a<1 is the fractional order. Train the regular quotient v_theta=u_theta/d^a=N_theta and use a weighted gradient loss that reflects the paper's boundary estimate.
Useful6/10
Difficulty4/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Train an unconstrained branch and a geometry-aware branch in parallel, then learn how much to trust the analytic branch. This preserves the benefit of explicit geometry on correctly specified tasks while allowing the model to ignore a misleading or irrelevant prior.
Useful6/10
Difficulty3/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Add a conservative correction after low-rank tensor compression so selected linear moments of an activation or learned state are exactly preserved. This can reduce tensor rank and memory without allowing compression error to accumulate in physically meaningful global quantities.
Useful6/10
Difficulty6/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a generic neural constitutive law or energy model with an ICNN that consumes the positive singular values of a deformation-like matrix and is convex and coordinatewise nondecreasing in those inputs. Train it as a lower approximation to a nonconvex target energy, so the network acts as a computationally cheap sufficient polyconvex-envelope surrogate rather than merely interpolating unstable samples.
Useful6/10
Difficulty4/10
Novelty4/10
✓✓ Beats tuned baseline
2026
Represent the PINN solution in a restricted polynomial or Taylor basis whose exponent set is supplied by tropical support analysis, instead of asking an MLP to discover the local series structure from scratch. The restriction removes coefficients that cannot occur in the formal solution, reducing trainable degrees of freedom and preventing spurious low-order or singular terms.
Useful6/10
Difficulty5/10
Novelty8/10
✓✓ Beats tuned baseline
2026
Replace an unconstrained feature vector entering a rational or resolvent-like neural operator by a polynomial feature whose first nonzero Taylor coefficient lies in a pole-safe subspace. For a pole of order m, the simplest guaranteed construction is psi(z)=(z-beta)^m v, which makes Q(z)psi(z) bounded even when Q(z) diverges. For lower-order cancellation, solve linear constraints among Taylor coefficients of psi so that all negative Laurent powers vanish.
Useful6/10
Difficulty5/10
Novelty8/10