Every idea extracted from recent arXiv mathematics papers — verified and unverified. Click an idea to open its full card; badges show the empirical verdict.
Replace Euclidean momentum for selected neural parameters with a mirror or Bregman update, while using the paper's accelerated Newton direction for the objective step. Entropy geometry is especially suitable for softmax MoE routers, while Euclidean or log-barrier geometries can be used for unconstrained or positive parameters.
Distill a teacher's attention into a student by matching sink mass and the normalized content distribution as separate targets rather than applying one KL divergence to the entire attention row. Use the Aitchison distance on the content composition, which compares relative token allocation and prevents a large common sink probability from overwhelming differences between content tokens.
Replace iid dropout or iid activation noise on spatial tokens with fluctuations generated by a conserved diffusing density. Each token receives a positive mass variable whose total mass is preserved, while Poissonian stochastic flux produces correlated perturbations that explore coherent local patterns rather than independently corrupting every feature. The density is autonomous and detached from autograd, so the regularizer adds little computational overhead.
Add global directed-curvature features to every node in a graph neural network or directed graph transformer. The features distinguish how a node functions as a source versus a destination in the graph's asymmetric metric, potentially exposing bottlenecks, hubs, sinks, and structurally central nodes that local message passing cannot identify.
Turn attention weights into a Boolean support scenario and prune edges using local-surjectivity constraints rather than independently thresholding each row. Preserve at least one compatible continuation for every local window, then favor a strongly connected support graph so pruning removes redundant mixtures while maintaining a globally coherent sparse attention pattern.
Replace a dense channel or token-mixing matrix with a product of positive bidiagonal factors, so information propagates through a controlled sequence of local couplings rather than arbitrary signed interactions. Initialize the factors from the paper's barycentric-subdivision factorization, then learn positive diagonal and off-diagonal parameters; the resulting map is structured, parameter-efficient, and constrained to remain totally positive.
Replace a free-form order-dependent gate with a positive mixture of Mellin powers $(1+s)^{-a}$. This gives a small, interpretable module whose response across the order variable is automatically generated by a positive measure and therefore inherits complete monotonicity, log-convexity, and Hankel-moment structure.
Use a learned asymmetric Finsler-like cost instead of the symmetric Euclidean distance in attention logits. The metric has a Riemannian quadratic part and a directional drift term, while a differentiable barrier enforces the strong-convexity condition derived for the paper's extended $(\alpha,\beta)$-metrics. This lets each attention head prefer one direction in feature space without producing pathological, non-convex distance landscapes.
Use the paper's q-ary overlap inequality as a regularizer for categorical neural networks. Two independently sampled attention, routing, or message-passing supports should rarely overlap in many locations; penalizing the moment q^{|S\cap S'|} discourages redundant histories and correlated interference between heads or experts.
Represent each of m neural branches by a positive input field f_i and a positive output field g_i, then penalize violations of the paper's multi-output Borell-Brascamp-Lieb bound at weighted barycenters. The constraint couples branches through both local normalized ratios and global mass ratios, encouraging calibrated multi-view predictions without requiring all output functions to be identical.
Replace scalar entropy penalties on attention maps with a matrix-valued heat-flow regularizer over a circular or periodic token coordinate. Each position stores a positive semidefinite matrix describing coupled heads, experts, or channels; heat smoothing is constrained by the sharp modified log-Sobolev and Bogoliubov–Kubo–Mori contraction rather than an arbitrary smoothing coefficient. This should suppress high-frequency routing noise while preserving positive matrix structure and reducing…