Every idea extracted from recent arXiv mathematics papers — verified and unverified. Click an idea to open its full card; badges show the empirical verdict.
Construct a sparse attention support by solving multiple small perturbed assignment problems between query and key embeddings and taking the union of the selected optimal matchings. Use the resulting spanning tree as the only set of cross-token edges, with edge biases determined by empirical assignment frequency.
Treat a recurrent or state-space layer as a finite-state Markov cocycle and constrain optimizer steps using the paper's inverse-logarithmic sensitivity of Lyapunov exponents near a zero exponent gap. Instead of enforcing a crude spectral-norm bound, allow updates that are harmless for long-run growth while shrinking steps that could substantially change the recurrent stability profile.
Replace the usual hand-designed expert-load penalty with a heterogeneous survival penalty derived from a susceptibility distribution. Each expert receives an availability factor q_e=G(A_e), where A_e is its cumulative recent routing pressure and G_e is a learned or fixed mixture of exponentials; highly used experts are suppressed smoothly, while heterogeneous experts can have different resistance to pressure. The mixture produces adaptive curvature and long-tailed penalties that may reduce…
Regularize a circular recurrent kernel by directly controlling the growth rate and phase velocity of its Fourier modes. This converts replay-speed selection into a low-dimensional spectral control problem and can suppress unstable or excessively slow modes without adding recurrent parameters.
Choose the consensus gain and gradient-tracking gain in decentralized training from the communication Laplacian spectrum rather than tuning them independently. The gains minimize the worst asymptotic pole radius for the paper's exact quadratic model, providing a principled initialization and a conservative stability safeguard for neural-network optimization.
Regularize a neural dynamical map so that its log-volume expansion is cohomologous to a constant rather than forcing the Jacobian determinant to be constant at every state. Learn a scalar potential that explains transient expansion and penalize only the non-telescoping component, which should reduce long-horizon gradient explosion or collapse while retaining useful average expansion.