Unverified
2026
Partition a neural network into heterogeneous parameter blocks or maintain several worker replicas, and model each block's optimizer state as a constrained linearized dynamical agent. At every synchronization interval, jointly optimize a finite sequence of parameter updates and a feasible common terminal parameter target, while enforcing consensus through distributed primal-dual iterations. Unlike ordinary gradient descent toward a fixed or implicit target, the target is selected together with…
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace the noisy Hutchinson estimate of a neural-network Hessian trace with a variance-reduced Hutch++ estimate computed only from Hessian-vector products. Use the estimated normalized curvature to cap or rescale the optimizer step, so learning-rate reductions occur when the loss landscape becomes globally sharp rather than when an individual minibatch gradient happens to be large.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Parameterize an orthogonal or semi-orthogonal neural weight matrix directly on the Stiefel manifold and update it with a Cayley retraction instead of unconstrained SGD plus a penalty or QR projection. The update preserves orthogonality exactly, is second-order accurate for the appropriate metric, and avoids the cubic QR factorization at every optimizer step.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Replace a dense token-mixing, MLP, or low-rank-adapter projection with a Bernoulli-signed sparse matrix normalized by the paper's predicted operator scale. Estimate the active representation dimension and use the effective scale to detect regimes in which extreme sparsity is likely to cause unstable amplification or dead rows.
Useful6/10
Difficulty4/10
Novelty5/10
Unverified
2026
Use two bounded self-adjoint transition operators in a recurrent or state-space block and penalize their normalized Hilbert--Schmidt commutator. When the penalty is small, the paper guarantees that the pair is close to exactly commuting operators, suggesting a controlled path to a shared eigenbasis and cheaper coordinate-wise dynamics. Add an optional numerical repair step that projects the learned pair toward a simultaneously diagonalizable pair.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Construct a generative or recurrent neural architecture with several contractive or mildly expanding branches, and explicitly control the geometric complexity of its invariant set using the sub-additive singular-value pressure of branch-Jacobian products. Instead of regularizing only the operator norm, the model can preserve anisotropic directions while targeting a desired attractor dimension, potentially improving coverage of structured data without uncontrolled folding or collapse.
Useful6/10
Difficulty6/10
Novelty8/10
Unverified
2026
Replace decentralized parameter averaging with consensus on cumulative local gradient states, followed by a barrier-FTRL update that stays strictly inside a convex feasible set. This is particularly suitable for federated learning with heterogeneous clients and for simplex-constrained mixture, router, or adapter parameters, where Euclidean projection can be unstable or expensive.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Replace a single global preconditioner for a multi-penalty neural objective with additive corrections adapted to the joint kernels of the penalty Jacobians. The optimizer is designed to remain effective when individual penalty weights change independently, avoiding the severe conditioning degradation that occurs when a correction space misses a singleton or partial joint kernel.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Replace an explicit recurrent transition with a learned descriptor relation in latent space, allowing some latent coordinates to satisfy algebraic constraints rather than being numerically integrated. Fit the relation using total-least-squares or iterative PCA on the jointly observed trajectory, so noise in every channel is treated symmetrically and the model can discover whether the latent system is index-0 or index-1.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Use one fewer point than the exact support threshold for a weighted multi-output linear head. The paper proves that the worst-case multiplicative loss penalty at this budget is only 1+1/(dm^2), giving a principled memory-saving option rather than an arbitrary subset-size heuristic.
Useful6/10
Difficulty4/10
Novelty4/10
Unverified
2026
Replace the recurrent transition or state-space mixer with a reversible transport followed by complementary relaxation of symmetric and antisymmetric feature components. The construction preserves a weighted energy and damps both parity sectors, giving bounded long-horizon powers without requiring the learned transition matrix itself to be symmetric. A numerical-range ellipse can be used as a cheap training-time certificate against transient growth.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Constrain the learned coefficients of a high-order linear recurrent or state-space layer using the block Hurwitz matrix associated with its matrix characteristic polynomial. Penalize near-singular Hurwitz blocks and, for degrees two and three, optionally enforce positive leading Hurwitz determinants; use companion-matrix eigenvalues as the definitive stability check rather than trusting determinant positivity at degree four or above.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
For a neural network predicting A coupled concentration or density fields, decompose Fourier-space fluctuations into a charge direction and its charge-neutral composition subspace. Hard-project the predicted fields to eliminate the global charge mode, and regularize their low-wavenumber covariance so that neutral modes retain finite susceptibility while the charge structure factor follows the Coulombic suppression S_ZZ(k) proportional to k squared. This should improve long-range physical…
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Initialize or regularize recurrent matrices so that each unit receives an approximately cancelling sum of positive and negative weights, while keeping the global variance and spectral radius fixed. Sweep a continuous balance parameter instead of imposing balance blindly, because the paper predicts qualitatively different behavior for saturating, sub-linear, and odd nonlinearities.
Useful6/10
Difficulty4/10
Novelty6/10
Unverified
2026
Train a neural state-space model whose one-step dynamics are linear in a fixed analytic feature vector, and use the empirical feature Gram matrix to detect whether passive trajectories identify the dynamics. Add data collection or replay only when the Gram matrix is poorly conditioned; the analytic-feature assumption predicts that persistent excitation should emerge without deliberately visiting every operating mode.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Apply the paper's quiver-semistability viewpoint to a graph-structured state-space layer, treating each node's latent state space as a quiver vertex and each message-passing or coupling matrix as an arrow. Penalize approximately invariant collections of node subspaces that receive little signal from the input, so the learned latent dynamics cannot hide useful information in unreachable subnetworks. A dual output-side penalty can prevent predictive information from becoming confined to…
Useful6/10
Difficulty5/10
Novelty8/10
Unverified
2026
Replace an unconstrained recurrent transition matrix with a J-selfadjoint matrix A, where J is a fixed diagonal signature matrix with only a small number of negative entries. Add a sampled Kreiss-resolvent penalty to suppress transient amplification while preserving the expressive dimension of the hidden state. The paper's bound predicts that worst finite-time amplification depends on the smaller inertia index rather than the full hidden dimension.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace the raw position/channel basis of a one-dimensional sequence module by eigenvectors of the projected cubic radial kernel matrix. Penalize or truncate coefficients in eigenmodes with many sign changes, giving a mathematically ordered smooth-to-oscillatory inductive bias while preserving the two-dimensional nullspace corresponding to affine trends.
Useful6/10
Difficulty4/10
Novelty6/10
Unverified
2026
Construct a graph-based latent state whose velocities evolve through free-flight updates and pairwise elastic collision operators. Each collision operator is orthogonal, so total latent kinetic energy is exactly conserved; a connected interaction graph is intended to eliminate unwanted component-wise polynomial invariants and improve long-horizon stability.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Construct a neural state-space model with an explicit first-order input-to-output path instead of forcing every output to depend only on deeply propagated hidden states. Penalize or reject learned linearizations whose transfer matrix has relative degree greater than one, then train a storage-function certificate for the remaining passive dynamics. This preserves the paper's relative-degree compatibility condition while allowing high-order internal memory.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Use two points with approximately equal minibatch loss to construct an ellipcenter: the intersection of the normal lines through the two points, where the normals are their gradients. The resulting update uses local curvature information in the span of two gradients and can be relaxed toward the current parameters or combined with momentum.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Build actuator or parameter constraints directly into the neural controller using a differentiable hard-saturation map rather than penalizing violations after the fact. This makes the Lyapunov certificate apply to the actual bounded controller and prevents training from exploiting unrealistically large actions.
Useful6/10
Difficulty3/10
Novelty4/10
Unverified
2026
Add a differentiable or inference-time projection to mesh and graph neural operators that contracts each predicted nodal state toward a weighted cell anchor. The anchor is the geometry-weighted mean, so the correction preserves the weighted integral exactly, while the contraction parameter is chosen to keep all nodal states inside a convex physical set such as positive density and energy or a probability simplex.
Useful6/10
Difficulty4/10
Novelty6/10
Unverified
2026
Replace an Erlang delay or exponential smoothing cascade in a recurrent or state-space layer by a positive rational kernel of the form \(\kappa(u)=C e^{-a u}p(u)^2\). Choose the degree-\(m\) polynomial by deleting the adjacent pair of Laguerre zeros with smallest relative gap from \(L_{m+2}\), then rescale the resulting density to unit mean. This preserves a nonnegative impulse response while reducing temporal jitter relative to Erlang filters.
Useful6/10
Difficulty6/10
Novelty7/10