△ Mechanism confirmed, baseline not beaten
2026
Use the truncated Fourier representation of an irregular domain as a reusable spectral mask inside an FFT convolution layer. This gives a cheap alternative to point-cloud neighborhood aggregation while explicitly suppressing contributions from outside the physical domain and improving behavior near corners, cusps, and holes.
Useful7/10
Difficulty5/10
Novelty6/10
✗ Mechanism failed
2026
Replace a dense coordinate-kernel interaction among N points by an orthogonal samplet transform with a sparse detail-detail matrix and a small polynomial branch. Detail basis vectors have vanishing moments, so smooth low-frequency behavior is represented by a few polynomial coefficients while localized residual interactions become sparse in the transformed domain.
Useful7/10
Difficulty6/10
Novelty7/10
✗ Mechanism failed
2026
Replace conventional nested bilevel optimization with simultaneous primal-dual updates that enforce inner-model stationarity through a Lagrange multiplier. Add quadratic dual regularization and projection onto a bounded ball, while estimating all Hessian-vector terms using finite differences of ordinary gradients.
Useful7/10
Difficulty5/10
Novelty6/10
✗ Mechanism failed
2026
Replace ordinary momentum with a semi-implicit velocity update containing viscous damping and a proximal dry-friction step, while evaluating the gradient at a look-ahead parameter point. The dry-friction proximal operator exactly zeros sufficiently small velocities, which may suppress late-training oscillations and create finite-time stationarity instead of merely asymptotic velocity decay.
Useful7/10
Difficulty4/10
Novelty7/10
✗ Mechanism failed
2026
Train on a sequence of Jin–Xin relaxation problems with decreasing relaxation width rather than training immediately on the singular conservation law. The network predicts both the conserved state and an auxiliary flux, and each stage is initialized from the previous stage so that the learned shock profile sharpens gradually.
Useful7/10
Difficulty5/10
Novelty7/10
✗ Mechanism failed
2026
Replace an explicit Euler residual update for a skew-coupled hidden state with a five-stage palindromic composition of exact shear maps. Use a=1/4, the unique real coefficient maximizing the analyzed spectral CFL interval, and adapt the step size from an estimate of the learned coupling operator's spectral norm.
Useful7/10
Difficulty5/10
Novelty6/10
✗ Mechanism failed
2026
Use Adam normally, but periodically estimate the spectrum of the Adam-preconditioned Hessian and add a damped low-rank Newton correction when the preconditioned curvature is strongly ill-conditioned or the gradient is concentrated in flat directions. The correction is computed only in a small Lanczos subspace, so the method targets cross-coupled ill-conditioning without materializing or inverting the full Hessian.
Useful7/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2026
Use a symplectic Hamiltonian update as a recurrent or state-space neural block, preserving a learned modified energy across many layers or time steps. This targets residual and recurrent architectures where ordinary Euler updates accumulate drift during long rollouts.
Useful6/10
Difficulty5/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Augment neural-network parameters with momentum variables and update the pair using a symplectic map generated by a Hamiltonian. The optimizer approximately preserves a modified Hamiltonian, reducing systematic energy drift and potentially making long unrolled optimization more stable.
Useful6/10
Difficulty4/10
Novelty4/10
✓✓ Beats tuned baseline
2026
Transform local neural residuals into the Ripa model's characteristic coordinates before spatial aggregation, apply a mode-dependent gate based on neighboring characteristic jumps, and transform back. This lets the model damp oscillatory acoustic or equilibrium-mode corrections near discontinuities without globally smoothing every feature.
Useful6/10
Difficulty6/10
Novelty6/10
✗ Mechanism failed
2026
Parameterize a learned feature-space operator as accretive but not necessarily symmetric, then apply its fractional power through a finite positive mixture of shifted resolvents. This provides a matrix-function layer that can represent directional and rotational interactions while avoiding unstable eigendecomposition of nonnormal matrices.
Useful6/10
Difficulty6/10
Novelty7/10
✗ Mechanism failed
2026
Replace a large dense layer whose input and output dimensions factor into multiple modes by a TT-matrix whose parameters are stored as a chain of small cores. Periodically apply TT-SVD rounding to remove weak singular directions and keep the representation within a prescribed approximation error. This transfers the paper's central computational principle—perform tensor-product contractions directly in compressed form—to neural network layers.
Useful6/10
Difficulty5/10
Novelty4/10
✓✓ Beats tuned baseline
2026
Replace the usual explicit residual update with a nonstandard general-linear block containing several internal feature stages. The effective step is a positive denominator function rather than the raw depth step, allowing the block to take large nominal steps while damping the update and preserving bounded activations. This is most promising for deep residual MLPs, neural ODE discretizations, and state-space sequence models where exploding hidden states limit usable depth.
Useful6/10
Difficulty6/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Separate a neural network into nonlinear hidden parameters and a linear output layer. Solve the output layer exactly by least squares, then update hidden parameters with a truncated-pseudoinverse Gauss-Newton step that discards numerically singular directions.
Useful6/10
Difficulty6/10
Novelty5/10
✗ Mechanism failed
2026
Replace explicit RK integration in a stiff neural ODE or continuous-depth residual network with the paper's stiffly accurate SDIRK4 discretization. Instead of performing a dense Newton solve for each implicit stage, solve the diagonal stage equation using a Chebyshev-accelerated residual iteration whose polynomial damps the negative, high-magnitude Jacobian modes responsible for stiffness.
Useful6/10
Difficulty7/10
Novelty7/10
✗ Mechanism failed
2026
Replace an unpreconditioned conjugate-gradient solve for a damped Gauss–Newton step with a two-level algebraic preconditioner derived from local Jacobian-row supports. Use overlapping local parameter blocks as Schwarz subdomains and a coarse basis containing low-energy local modes, so the optimizer can correct both localized and globally coupled parameter errors.
Useful6/10
Difficulty6/10
Novelty6/10
✗ Mechanism failed
2026
Replace a dense recurrent transition matrix with a periodic CMV-style product of alternating local 2x2 unitary cores. The transition is exactly norm-preserving, has O(n) trainable parameters under periodic tying, and can be applied through local factor operations rather than stored as an n-by-n matrix. Use turnover refactorization when changing the ordering or boundary connection of cores, enabling a compact cyclic unitary state-space layer.
Useful6/10
Difficulty5/10
Novelty5/10
✗ Mechanism failed
2026
Replace the usual linear predictor in continuation of an implicit neural state with a fractional-power predictor fitted from recent states, then correct the prediction using a pseudo-arclength constraint. This is designed for equilibrium layers, implicit sequence models, or homotopy training schedules where the state Jacobian becomes nearly singular and ordinary Newton correction or fixed-point iteration becomes unstable.
Useful6/10
Difficulty6/10
Novelty8/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a fixed or hand-tuned learning-rate schedule with a slowly exponentially increasing schedule, and restart the schedule whenever the update norm grows at least as fast as the schedule itself. The restart preserves the current parameters but resets the learning-rate multiplier, allowing the optimizer to repeatedly approach the largest locally stable step size without requiring a Hessian spectrum or a reliable initial learning-rate guess.
Useful6/10
Difficulty4/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Couple the number of operator training pairs to the output resolution instead of increasing the output grid independently. Refine the output discretization only while the oracle reconstruction improves, and increase the training set when the learned predictor remains substantially worse than the oracle decoder.
Useful6/10
Difficulty4/10
Novelty7/10
✗ Mechanism failed
2026
Replace a dense weight matrix by a cross approximation built from selected rows and columns rather than by a conventional truncated SVD. Periodically refresh the selected indices using residual leverage scores, warm-starting from the previous factorization so that the compressed layer can track weight changes during fine-tuning.
Useful6/10
Difficulty5/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Add a learned Riesz-transform branch that extracts normalized spatial gradients after diffusion by a positive parabolic operator. The diffusion branch carries smooth semantic content, while the Riesz branch represents boundaries, motion changes, and graph discontinuities. Resolvent smoothing makes the derivative branch less sensitive to feature noise than directly applying a finite difference.
Useful6/10
Difficulty5/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Add a fixed or weakly parameterized residual mixer whose interaction between sequence positions at distance \(r\) is proportional to \(1/(r\log^2 r)\). Instead of truncating the kernel at a short radius, represent its heavy tail with dyadic distance bands and compute each band using prefix sums or block pooling, giving every token access to arbitrarily distant context at roughly \(O(L\log L)\) cost.
Useful6/10
Difficulty5/10
Novelty6/10
✗ Mechanism failed
2026
Represent every mesh interface degree of freedom by one feature copy per incident cell, and apply local neural blocks directly to these cell tensors. Enforce inter-cell consistency with valence-weighted averaging only after selected layers or hierarchy transitions, avoiding repeated construction of a global sparse graph or assembled feature vector. This is suited to adaptive quadtrees, octrees, and finite-element neural operators.
Useful6/10
Difficulty5/10
Novelty7/10