Unverified
2026
Represent an iterative neural computation as a controlled dynamical system and learn sparse residual corrections that are active only for a finite prefix of iterations. Estimate local stable and anti-stable subspaces of the hidden-state Jacobian, increase the correction horizon only while the anti-stable component exceeds a tolerance, and force later controls to zero. This produces adaptive-depth inference with a quantitative stopping criterion.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Treat a small activation, gradient, or parameter perturbation as a seed and measure the number of newly affected downstream units or layers. Use the estimated branching ratio to control the optimizer step size or residual gains, keeping training in a subcritical regime where perturbation cascades have finite expected size instead of amplifying through the whole network.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace a raw minibatch gradient with an unbiased control-variate estimator that subtracts predictable components of per-example gradients and adds back their exactly or cheaply known population mean. Select the control-variate directions using leading eigenvectors of an online covariance operator, rather than using arbitrary scalar baselines. This should reduce gradient variance at fixed batch size and permit fewer examples per optimization step.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Modify decentralized parameter averaging or graph message passing so that each communication edge is classified using its observed delay and the spectrum of the instantaneous communication graph. Fast edges retain cooperative coupling, while excessively stale edges are attenuated or treated as antagonistic in a signed-Laplacian stability test. This should prevent a small number of very stale links from destabilizing otherwise stable asynchronous training.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace a purely diagonal or block-diagonal optimizer preconditioner with a truncated Woodbury correction selected in interaction coordinates. Per-example gradient combinations are ranked by their response through the base inverse preconditioner, so the retained directions are those most affected by curvature after normalization rather than merely those with the largest raw gradient norm.
Useful6/10
Difficulty6/10
Novelty4/10
Unverified
2026
Add a distributed spectral positional encoding to a graph neural network, graph transformer, sparse-attention model, or MoE router by computing the dominant eigenvector of the current weighted adjacency matrix with a few warm-started power iterations. Unlike a Fiedler-vector feature, this encoding uses only local neighbor aggregation, is naturally nonnegative for nonnegative adjacency weights, and can be updated incrementally when the graph or edge weights change.
Useful6/10
Difficulty4/10
Novelty4/10
Unverified
2026
Use the OT spectral bound as a conditioning signal for optimizing parameters of a neural cost or inverse-OT objective. Adapt the parameter step size and add a covariance floor whenever the estimated Jacobian lower bound collapses, preventing optimization from entering regions where Sinkhorn outputs become insensitive to the learned cost.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Search sparse reservoir wiring in graph space rather than repeatedly testing every candidate with its full nonlinear dynamics. Use graph descriptors to predict validation accuracy and nonlinear feature selectivity, then spend exact simulations on candidates with high predicted performance or high surrogate uncertainty.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace the fixed number of Sinkhorn iterations used to normalize an attention kernel with a per-example stopping rule based on the local spectral contraction factor. Estimate the remaining marginal error geometrically and stop early on easy examples while retaining extra iterations on difficult or nearly disconnected examples.
Useful6/10
Difficulty4/10
Novelty5/10
Unverified
2026
Track where the loss Hessian's eigenvectors are most sensitive to the current minibatch perturbation, rather than using only eigenvalues or a global learning-rate estimate. Apply extra damping only to spectral bands with high geometric response, allowing flat and well-separated curvature modes to retain a larger step size.
Useful6/10
Difficulty7/10
Novelty7/10
Unverified
2026
Use a low-degree residual polynomial of the neural-network Hessian rather than an interval-only Chebyshev polynomial, with the polynomial minimized over the bulk Hessian spectrum and isolated outlier eigenvalues simultaneously. The method should reduce oscillation caused by rare sharp directions without shrinking the learning rate for the bulk spectrum.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace the scalar learning rate of SGD or Adam's outer update by a blockwise Barzilai--Borwein estimate computed from consecutive parameters and gradients. Use gradient smoothing, denominator checks, and clipping so that the curvature estimate remains usable with stochastic neural-network gradients.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace a learned dense token-mixing matrix or residual-state transition with a sparse diffusive mixer whose Laplacian has a deliberately small largest Jordan block. Balance the two chain lengths around the central coupling/core, because the paper proves that this minimizes the worst defective transient among the tridiagonal family. Use a scalar residual step size to move the non-consensus spectrum inside the unit disk while preserving the sparse structure.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Implement f(A)b inside a neural network with a short Lanczos recurrence instead of an eigendecomposition or dense matrix-function operation. Use an SPD operator A such as a regularized graph Laplacian or feature covariance matrix, and choose the number of iterations by monitoring successive approximations. For Stieltjes functions, Lanczos is guaranteed to be close to the best vector in the same Krylov subspace.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace fixed-period federated averaging or distributed all-reduce with a Bernoulli communication decision whose probability is selected from estimated network connectivity and optimization conditioning. Local workers continue making corrected updates between communication events, while the contraction theorem exposes when communication is worth its cost.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Use enumerated weighing matrices as sparse orthogonal channel-mixing operators inside MLPs or residual blocks. Their ternary entries reduce multiplication to signed additions, while exact orthogonality prevents amplification or attenuation of feature norms; a trainable fixed-support version can recover expressivity without giving up computational sparsity.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace a recursive product implementation of a rational spectral filter with an additive sum of independently evaluated resolvents. Use the layer on a graph Laplacian, token-similarity operator, or other sparse feature operator to obtain a high-order filter without multiplicative roundoff and gradient amplification; the independent solves can also be batched or distributed across devices.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Construct a symmetric token affinity matrix and approximate only its spectrally outlying token-mixing modes using a small set of sampled landmark columns. The resulting low-rank operator replaces an \(O(n^2)\) dense mixer by two skinny matrix multiplications, while the paper's residual guarantee predicts that large-magnitude global interaction modes are preserved.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Construct one reference Lanczos basis for a symmetric propagation operator H, then derive several seed-specific spectral responses for Q_a(H)x_0 through finite-band polynomial connectors. With degree-r seeds, each transformed basis vector uses at most 2r+1 neighboring reference basis vectors, avoiding a separate Lanczos factorization for every seed.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace an ordinary dense or floating-point eigendecomposition of small Hessian or Fisher blocks with a sequence of rational Jacobi rotations. The rotations preserve Euclidean norms and can be stored using fixed-point coefficients, while approximately diagonalizing curvature so the optimizer can use separate coordinate-wise step sizes. This is especially relevant to low-precision training and blocks with mixed-sign curvature.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace an SVD-based pseudoinverse of a learned rectangular matrix with a low-degree polynomial initialization followed by a few Newton–Schulz iterations. The polynomial approximates the inverse Gram operator, while a cheap residual test accepts it only when the iteration is contractive and otherwise selects a conservative transpose-scaled initialization.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Track the dominant rank-r subspace of the gradient covariance online, then use that basis to construct a low-rank adaptive update or a controlled preconditioner. Unlike offline PCA refreshes, the Oja flow continuously follows changing training geometry while preserving orthonormality, potentially reducing the cost of second-order or Shampoo-like methods.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Add a selective redistribution branch to recurrent or graph propagation layers whose local Jacobian gains are too large. Instead of globally shrinking the layer, blend the unstable update at only the offending coordinates with a volume-weighted average of those coordinates and their upstream neighbors, using the paper's explicit threshold as the minimum stabilizing blend.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace slow primal-coordinate Gibbs sampling of a graph-structured Gaussian latent feature field with Gibbs sampling in the Fourier-dual factor graph. Use the primal convergence formula to determine when dual sampling is worthwhile, and map dual samples back to neural features through the graph incidence operator.
Useful6/10
Difficulty7/10
Novelty7/10