✗ Failed on benchmark
2026
Replace an instantaneous diagonal optimizer with a causal convolution of recent gradients, where cross-layer or cross-module gradient correlations define a finite-memory Onsager response matrix. Estimate the response at several parameter-block pairs and lags, integrate it to obtain a finite-time transport matrix, and use its regularized inverse or symmetric part to precondition the update. This targets optimization regimes in which gradients propagate between blocks with measurable delay, such…
Useful8/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Replace a slow sequence of resolvent or contractive fixed-point updates by a blockwise averaged-reflection extrapolation. The method computes reflected iterates R^j y_0, averages them with equal weights, and uses the result as the next macro-iterate. Unlike unconstrained Anderson acceleration, this construction has a uniform residual guarantee for every maximal monotone operator.
Useful8/10
Difficulty4/10
Novelty6/10
✗ Failed on benchmark
2026
Replace each recurrent neural state with two asymmetrically coupled variables: a slow state x_i and a fast momentum or drive variable v_i. Each coordinate or block updates independently using its locally available, possibly stale input; the auxiliary variable supplies inertia that suppresses harmful update-order sensitivity and can accelerate traversal toward a retrieved state or denoised solution.
Useful8/10
Difficulty5/10
Novelty6/10
✗ Failed on benchmark
2026
Replace recurrence or nearest-neighbour analogue lookup with a learned delay-coordinate observer that continuously corrects a latent state using the current observation. Constrain the observer's closed-loop Jacobian or linear state matrix to have spectral radius below one, so prediction error contracts geometrically and required burn-in grows logarithmically with target accuracy.
Useful8/10
Difficulty5/10
Novelty6/10
✗ Failed on benchmark
2026
For a recurrent or graph neural network with known local connectivity, estimate each node's local Jacobian row using only graph neighbors rather than all hidden coordinates. Use the resulting sparse Jacobian both to compute a contraction certificate and to regularize training toward dynamically local interactions, reducing estimator variance and the number of samples required for reliable stability decisions.
Useful8/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Replace an instantaneous largest-eigenvalue learning-rate ceiling with a delayed-instability monitor for a slowly ramped optimizer or network gain. When a dominant complex eigenpair crosses from negative to positive real part, permit a controlled post-crossing interval, but stop or roll back when the accumulated positive growth budget exceeds the perturbation/noise margin. This exploits slow-passage delay without allowing unbounded training instability.
Useful7/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
For a recurrent or implicit neural model driven by periodic inputs, solve for a periodic hidden-state orbit and continue that orbit as input amplitude or frequency changes. This replaces repeated cold starts from zero and should preserve convergence near parameter ranges where cold starts fail.
Useful7/10
Difficulty7/10
Novelty7/10
✗ Failed on benchmark
2026
Replace explicit zero-padding before FFT convolution by the paper's mixed-radix decomposition, which injects zeros through bounded tile sums and never allocates the padded input. The resulting transform is mathematically identical to the length-M transform of the explicitly padded signal, while reducing temporary storage and potentially memory bandwidth.
Useful7/10
Difficulty7/10
Novelty6/10
✗ Failed on benchmark
2026
Use trajectory sensitivities to remove neural units or parameter groups whose effects are redundant over the available data support. A parameter group is pruned when its Fisher contribution is small or its sensitivity is nearly collinear with other groups, producing a compact neural ODE without relying only on parameter magnitude.
Useful7/10
Difficulty5/10
Novelty8/10
✗ Failed on benchmark
2026
Treat a neural hidden-state process as a finite or discretized continuous-time Markov chain and define a target event as first entry into a target state set. Instead of estimating the derivative of the mean hitting time by expensive long rollouts, build an auxiliary regenerative chain that resets to the source state after reaching the target and estimate the same response from its stationary distribution. Penalize disagreement between this response prediction and short empirical perturbation…
Useful7/10
Difficulty6/10
Novelty8/10
✗ Failed on benchmark
2026
Replace the naive pseudospectral evaluation of a quadratic neural-operator nonlinearity with a two-point split-form product. Use the entropy-stable (alpha, beta) = (1/3, 2/3) split as the default, or learn alpha under the consistency constraint alpha + beta = 1 while monitoring energy growth. The goal is to suppress weakly underresolved aliasing and prevent long-horizon rollout blow-up without full 2/3-rule zero-padding.
Useful7/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Replace magnitude pruning in a trained recurrent network with stochastic pruning probabilities computed from weight magnitudes and the covariance of neuron activities under injected noise. Connections whose endpoints fluctuate in a sign-compatible way receive higher retention probability, while retained weights are rescaled to preserve average recurrent strength. The method uses local weights and activity covariance, avoiding Hessian construction and expensive global saliency optimization.
Useful7/10
Difficulty4/10
Novelty7/10
✗ Failed on benchmark
2026
Use the paper's sharp sK approximately equal to 1 phase transition to choose between conservative Fejer averaging and higher-order polynomial filtering. When the local fixed-point spectrum is separated from eigenvalue 1, use a Jackson-type filter; near the critical regime, use the safe Fejer filter instead of unrestricted Anderson extrapolation.
Useful7/10
Difficulty7/10
Novelty7/10
✗ Failed on benchmark
2026
Split hidden dynamics into relaxation bands when the Jacobian spectrum has a gap, evolve each band with its own timescale, and retain an explicit cross-band exchange term. This yields a principled dual-timescale RNN or SSM rather than choosing fast and slow branches heuristically.
Useful7/10
Difficulty6/10
Novelty6/10
✗ Failed on benchmark
2026
Replace repeated time-stepping of a stiff linear state-space block with a quadrature approximation to its inverse Laplace transform. The layer propagates a hidden state using a small set of complex shifted linear solves, which can be batched and reused across many time steps or parameter values.
Useful7/10
Difficulty6/10
Novelty6/10
✗ Failed on benchmark
2026
Reparameterize a recurrent or state-space layer so that its hidden-state update contains an explicit stabilizing feedback controller, while the neural network learns only a residual control in the feedback coordinates. Choose K to reduce finite-horizon state-propagation amplification, suppressing exploding hidden states and gradients on long sequences.
Useful7/10
Difficulty5/10
Novelty5/10
✗ Failed on benchmark
2026
Replace quadratic self-attention over a sequence with a bank of K auxiliary exponentially decaying states whose rates are fitted directly from the empirical autocorrelation of the sequence features. Each mode captures a distinct time scale, so the module can represent short- and long-range dependencies with O(TK) computation and O(K) recurrent memory rather than storing all previous tokens. Constrain decay rates to be positive and use the paper's extended Markovian block structure to obtain a…
Useful7/10
Difficulty5/10
Novelty4/10
✗ Failed on benchmark
2026
Replace a weight-tied residual or neural-ODE stepper with an explicit Runge–Kutta method satisfying the reused-last-stage conditions. The final derivative is evaluated at the exact endpoint and becomes the first derivative of the next step, saving one expensive neural-vector-field call per step while preserving the designed integration order.
Useful7/10
Difficulty5/10
Novelty5/10
✗ Failed on benchmark
2026
Regularize a circular recurrent kernel by directly controlling the growth rate and phase velocity of its Fourier modes. This converts replay-speed selection into a low-dimensional spectral control problem and can suppress unstable or excessively slow modes without adding recurrent parameters.
Useful6/10
Difficulty6/10
Novelty8/10