Every idea extracted from recent arXiv mathematics papers — verified and unverified. Click an idea to open its full card; badges show the empirical verdict.
Replace dense attention over structured object tokens with attention over role-filler tensor-product representations. A learned query specifies both a role and a filler, retrieves objects matching that binding, extracts a target role, and rebinds the extracted filler into an output object.
Estimate the largest certified input perturbation radius for a neural network using nested reduced primal and dual linear programs rather than solving the complete verification LP immediately. The primal sequence gives certified feasible robustness reserves, while the dual sequence gives valid upper bounds; verification may stop as soon as the interval width is below a prescribed tolerance.
Replace penalty-based orthogonality training for an \(n\times k\) weight or feature matrix \(X\) with a projected spectral flow driven by a symmetric matrix \(A\), such as a minibatch covariance or task-derived curvature estimate. The update rotates the subspace toward the top or bottom eigenspaces while preserving \(X^{\top}X=I_k\) through QR or Cayley retraction, avoiding the ill-conditioning caused by large orthogonality penalties.
Replace the global EMA update for each linear-layer momentum matrix with a delta-rule update that learns the current output-side gradient value only along the current input-key direction. Frequently occurring directions are corrected repeatedly, while rarely visited directions are not unnecessarily overwritten or uniformly decayed. Use the resulting matrix as the ordinary momentum buffer in SGD, AdamW, or another optimizer.
Replace the usual sum of pairwise modality similarities with a higher-order score based on the coordinatewise Hadamard product of all normalized modality embeddings. For modalities indexed by i=1,...,m, score a tuple using s(x_1,...,x_m)=\omega^\top(\bar g_1(x_1)\odot\cdots\odot\bar g_m(x_m)), where \omega is learned and \odot is coordinatewise multiplication. This adds explicit m-way interactions without concatenating raw features or introducing a joint encoder.
Replace a dense neural-network weight tensor with a graph tensor network whose physical modes and internal edge ranks are specified by a sparse rank-adjacency matrix. Unlike tensor-train or hierarchical Tucker layers, the graph can contain selected cycles and skip connections between tensor modes, allowing the factorization topology to match correlations in the weight tensor. Fit the layer with GTN-SVD at a prescribed tolerance and compare accuracy, parameter count, and tensor-contraction…
Add a low-dimensional dynamical observer to every graph-NN or mixture-of-experts communication link and estimate additive message corruption before aggregation. The observer is switched together with the network mode, such as changing adjacency, expert assignment, attention mask, or operating regime; the corrected message is the received message minus the estimated attack. This should remain effective against attacks with arbitrarily large amplitude if their temporal rate is bounded and the…
Replace an unconstrained deep RoPE attention residual update by a spherical or norm-preserving update whose attention kernel has a known positive floor. Estimate the reversible transverse spectrum of the current attention matrix and choose the residual step size below its explicit Euler stability limit; use the angular token diameter as a runtime contraction monitor.
Replace full-state quantized write-back in a deep low-bit residual stack with quantized increment error feedback. The residual branch quantizes the proposed increment after adding the previous carry, while the carry stores the exact discrepancy; this makes the total error telescope instead of accumulating approximately once per layer.
Use fast memory as read-only scratch state during the internal pondering iterations of a recurrent block, and apply memory writes only after the latent computation has halted or crossed a write gate. This prevents the transition operator from changing while it is being iterated, reducing self-corruption of the evidence used for subsequent reasoning.
Before quantizing a matrix product, reparameterize its factors as A'=AT and B'=T^{-1}B, preserving the exact full-precision product while changing the quantization difficulty of each factor. Choose a positive diagonal T=diag(t_1,...,t_K) that minimizes predicted post-quantization product error, rather than using output-channel scaling or a fixed heuristic grid. The gauge can be shared across several products when transformed-copy cost matters.
Use an online estimate of the positive feedback gain among logits, routing probabilities, and representations to adjust the softmax temperature. Increase temperature when the estimated cyclic gain approaches the instability regime, preventing exponential amplification and router collapse without globally weakening all layers.
Replace the usual random or elementwise-positive linear-attention feature map with a rank-one positive-semidefinite feature map derived from query and key vectors. For normalized inputs, the resulting kernel is the squared inner product, which is nonnegative and gives a geometrically structured interference pattern that is better suited to associative recall than an arbitrary low-rank feature map.
Augment a neural latent or sequence model with a Gaussian behavior head that predicts an entire future trajectory jointly from the observed prefix and planned inputs. Instead of recursively applying only a point predictor, condition the learned joint trajectory covariance on the available prefix, producing a corrected future mean and uncertainty that incorporates temporal correlations.
Replace layer-local reconstruction in post-training quantization with a sequential objective that explicitly cancels the error already accumulated by the quantized prefix. For each layer, quantize its weights so that its local residual approximately negates the propagated incoming deviation, preserving the teacher trajectory even when the codebook is binary or 4-bit.
Compress each layer's KV tensor with a partial Tucker approximation over token and feature axes, then encode the truncation residual with a rotated uniform quantizer. Select token rank, feature rank, and residual bit-width jointly under a global byte budget, allowing values with flat spectra to receive residual bits while keys may receive more low-rank capacity.
Replace unconstrained residual updates with blocks whose Jacobian is monitored through a Davis–Wielandt shell. The shell simultaneously measures directional dissipation and non-normal amplification, yielding a per-block step-size or residual-scale bound that is stronger than checking only the largest eigenvalue or spectral norm.
Compress each hidden layer by retaining directions that are simultaneously reachable from the observed input distribution and observable at the network output. Unlike PCA or SVD, the retained subspace is weighted by downstream task sensitivity, so high-variance but output-irrelevant directions can be removed while low-variance predictive directions are preserved.
Replace an unconstrained token mixer or feed-forward residual map with a feature-conditioned rational operator whose transfer matrix is positive real on the positive orthant. Apply it through a resolvent, rather than an additive residual, so that the accretivity certificate yields a nonexpansive implicit update and suppresses activation explosions.
Replace ordinary row-softmax attention with a doubly stochastic Sinkhorn attention plan W, and periodically recover a gauge-fixed pairwise cost from W using the exact double-centering identity. Use this recovered cost to initialize or regularize a structured attention score, making the attention geometry identifiable despite arbitrary query and key row and column offsets.
Partition a neural network into interacting modules and constrain the product of their local finite-region gains and coupling strengths so that the resulting gain matrix has spectral radius below one. This transfers the paper's small-gain-like mechanism and gives a quantitative large-signal boundary: instability or exploding activations should emerge as the spectral radius approaches one, while a weighted Lyapunov function should contract below that boundary.
Replace dense graph self-attention with two parallel branches: exact softmax attention only over graph neighbors and a global linear-attention branch that summarizes all nodes through feature-space statistics. A learned node-wise gate interpolates between the branches, allowing locally structured nodes to use sparse attention while retaining a global-information path.
Replace Muon's single momentum matrix with a weighted mixture of fast and slow relaxation modes. The fast mode tracks rapidly changing gradients while the slow mode preserves a longer-horizon direction; their mixture is semi-orthogonalized and applied as the matrix update.
Augment Adam with a layerwise stability monitor based on the paper's normalized frozen stability parameter. Estimate each layer's local sharpness and reduce that layer's learning rate whenever c eta S divided by sqrt(v)+epsilon approaches or exceeds 2. This directly tests whether the one-dimensional edge-of-stability boundary is useful as a safety controller in practical neural-network training.