△ Mechanism confirmed, baseline not beaten
2026
Replace independent architecture generation with a diffusion mutation kernel that starts from a known valid neural architecture, re-noises it for only a fraction of the diffusion horizon, and denoises it conditionally toward a new architecture. The resulting candidates should remain closer to the parent and retain validity at low mutation strength, while larger re-noising fractions should produce greater novelty and access to distinct architectural basins.
Useful8/10
Difficulty6/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Build a shallow neural model whose input at every discretization level is embedded into a common Hilbert space with uniformly bounded norm, and constrain every neuron parameter in the corresponding dual norm. The statistical complexity then depends on the Hilbert norm bound rather than the number of retained coordinates, allowing one model design to operate across increasingly fine measurements.
Useful8/10
Difficulty4/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace the usual isotropic Gaussian random Fourier features with a frequency distribution matched to the expected spectral regularity of the target function. For coordinate fields, operator-learning maps, or PDE solution surrogates, this should place more features where the target Fourier energy lies and improve approximation at the same feature count. Stabilize the resulting feature matrix with whitening or ridge regression because spectral accuracy can create severe ill-conditioning.
Useful7/10
Difficulty4/10
Novelty5/10
✗ Mechanism failed
2026
Replace a trainable shallow MLP hidden layer by a frozen bank of smooth sigmoid ridge functions and train only a linear output head. Choose the feature count and parameter sampling regime using the theorem's explicit dependence on input dimension d, target regularity k, evaluation norm m, and confidence delta. The construction is especially appropriate for smooth regression, scientific surrogate models, and PINNs, where derivatives of the network output are part of the loss.
Useful7/10
Difficulty3/10
Novelty5/10
✗ Failed on benchmark
2026
Replace deterministic binary-tree pooling or hierarchical feature aggregation by a stochastic merge that chooses either elementwise addition or elementwise minimum. The mixing probability p controls whether zero or sparse states proliferate or disappear, with a predicted absorbing-state transition at p = 1/2.
Useful7/10
Difficulty5/10
Novelty8/10
△ Mechanism confirmed, baseline not beaten
2026
Train a neural network to represent an elliptic solution using Walk-on-Spheres rollouts as stochastic targets instead of evaluating a mesh-based PDE residual. For each input point, recursively jump to a random point on the largest interior sphere, accumulate source contributions, evaluate boundary data at termination, and regress the network output to the resulting Monte Carlo estimate.
Useful7/10
Difficulty5/10
Novelty6/10
✗ Failed on benchmark
2026
Add a conditional-law head that maps a compact representation of an initial distribution and a shared-noise trajectory to a Gaussian mixture, then computes downstream predictions as analytic expectations under that mixture. This can replace expensive particle rollouts or particle pooling in stochastic world models and conditional diffusion systems while retaining multimodality.
Useful7/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Replace a single optimizer trajectory by N parameter particles and optimize the time until the first particle reaches a target loss or reward threshold. Use distinct interaction regimes: bounded normalized interactions should provide only the usual logarithmic extreme-search improvement, whereas unnormalized coherent force accumulation and stochastic pairwise kicks should produce distinct 1/N and 1/(N ln N) first-hit laws.
Useful7/10
Difficulty5/10
Novelty7/10
✗ Mechanism failed
2026
Use neural networks to estimate outcome and treatment nuisances, then edit the resulting debiasing weights so that residualized treatment is conditionally orthogonal to an adversarial class of covariate functions. This should reduce coefficient bias when the two nuisance networks have strongly imbalanced approximation errors, without requiring either network to be correctly specified.
Useful7/10
Difficulty5/10
Novelty7/10
Unverified
2026
For bandlimited or progressively higher-frequency targets, construct a nested random Fourier layer whose frequency window grows according to the paper's bandwidth laws instead of sampling all frequencies from one fixed range. Train with a low-bandwidth subset first, then activate additional frequency blocks. This creates a mathematically motivated spectral curriculum and may reduce early optimization difficulty while preserving high-frequency accuracy.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace a finite-dimensional latent prior for fields or trajectories with a continuous flow acting on function coefficients in a separable Hilbert space. A Gaussian reference with an explicit covariance spectrum is transformed by a learned vector field, so the model can represent smooth, non-isotropic function distributions while sharing parameters across discretization resolutions.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace a dense Gaussian random-feature layer by a sparse Poisson-chaos layer with the same learned kernel, and penalize the layer's normalized fourth add-one energy. The penalty suppresses rare single-atom jumps, so the sparse layer approaches the Gaussian same-kernel representation in Wasserstein distance while preserving computational advantages from sparse Poisson sampling.
Useful6/10
Difficulty6/10
Novelty8/10
Unverified
2026
Construct training and evaluation examples whose every (d-1)-variable marginal is exactly independent, but whose full d-variable distribution contains a parity interaction. This isolates genuine high-order reasoning from shortcuts based on pairwise or lower-order statistics and can expose whether attention or MLP architectures learn the intended interaction.
Useful6/10
Difficulty3/10
Novelty6/10
Unverified
2026
Reparameterize all Bayesian neural-network weights by a bounded latent vector in the unit ball and use a simple ball-constrained MCMC kernel instead of unconstrained HMC or random-walk sampling. A radial diffeomorphism expands points near the ball boundary into arbitrarily large weights, preserving heavy-tailed posterior mass while preventing the sampler from numerically wandering through an unbounded parameter space.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Generate many random symmetric decision trees and encode each input by the one-hot indicator of its reached leaf. Use the resulting fixed random feature vector as an additional input to an MLP, or train only a ridge/linear prediction head on it. The tree ensemble's Gaussian-process-limit interpretation predicts that increasing the number of independent trees should approximate a stable kernel while avoiding MCMC and difficult optimization over discrete split structures.
Useful6/10
Difficulty4/10
Novelty7/10
Unverified
2026
Replace a dense spatial parameter field in a neural field or convolutional adapter by a truncated squared-exponential KL expansion with analytic Gaussian-Hermite modes. The amplitude and correlation length remain trainable, but changing them only rescales coefficients and basis parameters instead of triggering a numerical eigensolve.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Use nested parameter-confidence sets to control how far a neural optimizer may move when its local loss dynamics are uncertain. Estimate a local linear model of parameter or gradient evolution, propagate a homothetic tube for possible next iterates, and impose a trust-region radius that shrinks when the estimated contraction margin is insufficient. This gives a model-based alternative to heuristic gradient clipping and predicts a sharp learning-rate boundary tied to the largest uncertain…
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Train polynomial interaction features in increasing Hermite degree and activate a new degree only when the previous spectral shell is fitted. This turns the paper's spectral approximation behavior into a curriculum and explicit regularizer, preventing high-order interaction parameters from amplifying noise before the low-order Gaussian structure is learned.
Useful6/10
Difficulty4/10
Novelty8/10
✗ Mechanism failed
2026
Construct a differentiable approximate projection layer for a convex feasible set by averaging Gaussian perturbations conditioned on feasibility. Unlike hard projection or penalty methods, the output is smooth in the input and inherits firm nonexpansiveness in the convex case.
Useful6/10
Difficulty4/10
Novelty5/10
✗ Mechanism failed
2026
Replace the first dense layer on q-ary categorical features by a Fourier interaction layer containing only monomials whose coordinate support is at most s. Use a Bohnenblust–Hille-inspired quasi-norm on coefficients, separately for each interaction order, to prevent a small number of high-order interactions from dominating the output. The resulting model has an explicit interaction-order knob and can be tested against a dense MLP at matched parameter count.
Useful6/10
Difficulty5/10
Novelty7/10
✗ Mechanism failed
2026
Replace an unconstrained high-order polynomial interaction module with features generated by Gaussian matrix contractions and their exact Wick expansion. The resulting interactions are sums of products of power-sum invariants, with coefficients fixed by perfect-matching counts, providing a low-parameter inductive bias for permutation- or orthogonal-structured data.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Add a calibrated robustification rule after a symmetric polynomial feature map z(x)=vec(x^{\otimes d}). For a convex Lipschitz head or loss applied to z(x), compute a high-probability deviation radius from the paper's concentration rate and clip only examples beyond that radius. This explicitly accounts for the large radial fluctuations created by reusing the same vector in every tensor slot.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Add an auxiliary loss that makes selected representation coordinates insensitive to all subsets of fewer than d variables while retaining a d-way parity statistic. The objective discourages the network from solving a task through pairwise shortcuts and explicitly rewards a controlled high-order interaction.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace a fixed quantile output with a Lambda-quantile head that receives a predictive sample set and applies a learned value-dependent threshold \(\Lambda(x)\). Unlike ordinary quantile regression, the model can use a low threshold in one value range and a high threshold in another, which is useful when error costs or calibration requirements vary across the output domain. Start with a piecewise-constant or monotone spline parameterization, then test whether allowing controlled…
Useful5/10
Difficulty5/10
Novelty6/10