✗ Failed on benchmark
2026
Replace a fixed soft-threshold, ReLU-like gate, or manually chosen activation shrinkage with a monotone learned shrinkage function fitted by an observed-data quadratic-risk criterion. The gate can interpolate between identity, ridge-like attenuation, hard thresholding, and lasso-like soft thresholding, allowing each layer or channel group to adapt its bias–variance tradeoff from the current minibatch.
Useful6/10
Difficulty5/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a binary classifier's unconstrained final logit with a differentiable likelihood-ratio head based on two squared Mahalanobis radii in a learned embedding space. Approximate the shared radial generator with a small fractional-power basis, allowing the head to model heavy-tailed class geometry that an affine QDA logit cannot represent while remaining much smaller than a generic nonlinear head.
Useful6/10
Difficulty5/10
Novelty6/10
✗ Mechanism failed
2026
Replace ordinary hidden-weight decay with a recursive ℓ1 variation penalty on the coefficients used to combine activated functions from the previous layer. Use normalized activations \(\sigma_s(t)=\sigma(st)/s\) so that the learned scale parameter \(s\) controls feature shape separately from the coefficient magnitude charged by the variation norm.
Useful6/10
Difficulty5/10
Novelty5/10
✗ Mechanism failed
2026
Replace an unconstrained scalar activation or nonnegative gate with a finite positive mixture of rational Bernstein basis functions. The learned function is monotone and concave on the nonnegative half-line, while its derivatives have controlled alternating signs; this can prevent pathological feature amplification and gives an interpretable shape prior. Use the paper's sharp exponent restriction τ≤1/2 rather than treating the power as an arbitrary hyperparameter.
Useful6/10
Difficulty4/10
Novelty7/10
✗ Mechanism failed
2026
Replace a dense channel or token-mixing matrix with a product of positive bidiagonal factors, so information propagates through a controlled sequence of local couplings rather than arbitrary signed interactions. Initialize the factors from the paper's barycentric-subdivision factorization, then learn positive diagonal and off-diagonal parameters; the resulting map is structured, parameter-efficient, and constrained to remain totally positive.
Useful6/10
Difficulty5/10
Novelty8/10
✓✓ Beats tuned baseline
2026
Train an unconstrained branch and a geometry-aware branch in parallel, then learn how much to trust the analytic branch. This preserves the benefit of explicit geometry on correctly specified tasks while allowing the model to ignore a misleading or irrelevant prior.
Useful6/10
Difficulty3/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Replace random Fourier features or a dense sinusoidal positional encoding with a compact bank whose frequencies are the continued-fraction denominators of an irrational number. Inverse-frequency amplitudes provide multiscale structure with a controlled sub-Lipschitz regularity profile, while lacunarity reduces the number of frequencies needed to represent oscillatory structure.
Useful6/10
Difficulty3/10
Novelty5/10
✗ Mechanism failed
2026
Replace an unconstrained high-order polynomial interaction module with features generated by Gaussian matrix contractions and their exact Wick expansion. The resulting interactions are sums of products of power-sum invariants, with coefficients fixed by perfect-matching counts, providing a low-parameter inductive bias for permutation- or orthogonal-structured data.
Useful6/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Replace ordinary Jacobian penalties in coordinate MLPs or deformation networks with a learned local rotation frame and a polyconvex energy of the relative stretch. Penalize \(U\), its cofactor, and its determinant through a convex function, while separately smoothing the rotation field through \(R^T\operatorname{Curl}R\). The intended benefit is resistance to fold formation and better conditioning than directly penalizing \(\|J-I\|^2\), especially for large deformations.
Useful6/10
Difficulty6/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Use an approximate decision diagram to select a structured subset of neurons, channels, attention heads, or attention edges when their quadratic interactions are sparse or inverse-sparse. Merge states that agree on a local interaction boundary and accept a tunable epsilon loss in the pruning objective, obtaining a representation whose size is linear in model width for fixed accuracy tolerance.
Useful6/10
Difficulty6/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Replace one deterministic residual update with a short cyclic composition of learned vector fields evaluated for randomized, short run times. Because finite compositions of noncommuting flows generate directional-derivative and Lie-bracket terms, changing the cycle order gives the network an explicit, low-cost way to learn drift directions that are unavailable from the individual vector fields alone.
Useful6/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2025
Replace a conventional scalar activation by a geometrically indexed family of affine pieces whose slope changes with the logarithmic magnitude of the input. The same two endpoint parameters are reused across all scales, giving a compact, explicitly scale-aware activation that can represent different responses for exponentially separated activation magnitudes.
Useful6/10
Difficulty4/10
Novelty7/10
Unverified
2026
Replace Fourier or sinusoidal one-dimensional coordinate features with a trainable radical layer f(x)=sum_i c_i sqrt(P_i(x)), where every P_i is a strictly positive quadratic. A nonzero scalar output formed by such a layer has at most 2n distinct real zeros, providing an explicit bound on sign changes and suppressing uncontrolled ringing. Use the radical features as an input embedding for a conventional MLP or neural implicit field.
Useful5/10
Difficulty4/10
Novelty8/10
Unverified
2026
Prune higher-order tensor or polynomial interactions while preserving an exchange-closed support instead of independently retaining the largest weights. The resulting sparse interaction pattern avoids arbitrary holes and retains structured substitutions between coordinates, potentially improving parameter efficiency and robustness after pruning.
Useful5/10
Difficulty5/10
Novelty9/10
Unverified
2026
Represent an original neural-network block and a proposed rewritten block as constrained optimization formulations over inputs and trainable parameters, then certify that the rewrite preserves feasibility and the ordering of losses over a bounded domain. This gives a compiler or pruning pipeline a formal reject/accept gate instead of relying only on numerical regression tests.
Useful5/10
Difficulty7/10
Novelty8/10
Unverified
2026
Regularize a scalar network output so that its superlevel sets are approximately quasiconcave in input or latent space. Instead of penalizing the full Hessian, penalize positive curvature only in directions orthogonal to the output gradient, matching the paper's projected-Hessian and weighted 1-Laplacian structure.
Useful5/10
Difficulty6/10
Novelty6/10
Unverified
2026
Treat a finite shallow network as a discrete signed measure over ridge atoms and grow or prune neurons according to their contribution to total variation. This turns width selection into an atomic approximation procedure: add neurons correlated with the current residual and remove coefficients that consume budget without contributing materially.
Useful5/10
Difficulty5/10
Novelty5/10
Unverified
2026
Constrain a channel-mixing layer to be a product of nonnegative bidiagonal matrices, rather than an unconstrained dense matrix. The resulting totally nonnegative operator is predicted not to increase sign oscillations in ordered channel features, potentially reducing high-frequency feature noise and making deep stacks more stable.
Useful5/10
Difficulty4/10
Novelty8/10
Unverified
2026
Add a differentiable penalty that encourages a neural implicit field to have a controlled local homogeneity degree across concentric spatial scales. The penalty compares the flux-normalized frequency at adjacent radii, optionally targeting a desired degree k, so the network is discouraged from producing scale-inconsistent or oscillatory local geometry.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace two unconstrained scalar quadratic feature heads with a pair whose quadratic forms admit a positive-definite linear combination. This prevents the two heads from simultaneously vanishing on any nonzero hidden vector, which can reduce representation collapse and improve the conditioning of downstream gates or auxiliary objectives. The constraint can be implemented softly with a spectral-margin penalty, or exactly by parameterizing one learned pencil as positive definite.
Useful5/10
Difficulty4/10
Novelty8/10
Unverified
2026
Replace raw powers or unconstrained polynomial spectral features with normalized Jacobi features whose amplitude is provably bounded on the entire input interval. Use trainable mixtures of these features in a positional encoding, graph spectral layer, or MLP front end, while preserving the theorem's normalization and optionally constraining the learned mixture norm.
Useful5/10
Difficulty4/10
Novelty7/10
Unverified
2026
Replace a hard nonnegative slack or ReLU output by the barrier-derived map \(x_s(w)=\frac{w+\sqrt{w^2+4s}}{2}\). Unlike an arbitrary smooth activation, this output is the unique positive solution of \(x(x-w)=s\), so the network can explicitly monitor complementarity and anneal \(s\) toward the true inequality-constrained solution. Use it in a constrained output head or in hidden layers whose activations represent nonnegative resource, probability, or routing slack variables.
Useful5/10
Difficulty3/10
Novelty4/10
Unverified
2026
Replace raw polynomial features in a scalar MLP expansion with endpoint-weighted orthonormal Jacobi features. The paper's envelope gives a degree- and parameter-aware scale for each feature, preventing high-degree terms or endpoint behavior from dominating gradients while preserving a richer approximation basis than low-degree monomials.
Useful5/10
Difficulty4/10
Novelty6/10
Unverified
2026
Add coefficient-spreading and moment-calibration mechanisms to binary or sign-noised linear layers. For each output neuron, normalize its real-valued weights and penalize large normalized coordinates, so its signed preactivation obeys the paper's quadratic Gaussian approximation rather than the weaker linear bound. This should make activation scales more predictable and reduce training instability caused by highly concentrated binary projections.
Useful5/10
Difficulty3/10
Novelty4/10