Every idea extracted from recent arXiv mathematics papers — verified and unverified. Click an idea to open its full card; badges show the empirical verdict.
Estimate localized directional Fourier correlations between intermediate activations and their loss residuals, then penalize anisotropic concentration. The method can discourage unstable feature directions and improve robustness without requiring a full microlocal distribution reconstruction.
Insert a Walsh-Hadamard transform before a quantized categorical or activation bottleneck and assign coordinate-dependent quantization precision using the attenuation spectrum of a quaternary symmetric noise model. Coordinates corresponding to tensor-product frequencies with many nonzero indices are attenuated by higher powers of \(\delta\), so their quantization can be made coarser with little effect on the reconstructed post-noise representation. This creates a structured, fast transform…
Replace a large flat positional-embedding table with a recursively decoded nine-way address whose child transformations contract coordinates by exactly 1/3. Encode an input position using features attached to the address prefix at several depths, guaranteeing that increasing depth produces a geometrically localized representation and that an infinite valid address cannot ambiguously represent two distinct points. This is especially suitable for 2D vision tokens, maps, point clouds, or…
Replace a dense third-order channel-interaction tensor by a fixed sparse support selected through the paper's uniform-marginal infeasibility certificate. Supports with a large dual margin have an effective entropy base below the channel alphabet size, suggesting fewer independent interaction slices and cheaper contractions. Use the certificate either during architecture search or as a pruning score for an already-trained tensorized layer.
Build a neural feature-mixing layer from a truncated shift S and a diagonal phase operator T satisfying TS=qST, with |q|=1. The relation forces moving one position in the graded feature basis to multiply the phase operator by q, providing a compact inductive bias for periodic, phase-sensitive, or cyclic data.
Add a low-dimensional spectral regularizer to an encoder or transformer representation by estimating the first N nonconstant modes of its Gaussian-weighted diffusion operator. Penalize excessive reciprocal spectral mass and unequal low-frequency eigenvalues, using a Gaussian-ball reference calibrated to the representation's effective mass; this discourages latent directions from becoming weak, collapsed, or strongly anisotropic.
Add an integer-lattice feasibility layer after ordinary top-1 or top-2 MoE routing. The router first produces its usual expert assignments, then minimally changes a small number of low-confidence assignments so the batch count vector lies in a prescribed lattice or desired coset, eliminating persistent modular load imbalance that ordinary auxiliary losses may not detect.
Replace raw hyperbolic embedding-radius regularization with a dimension-aware effective-radius target. For embeddings concentrated near hyperbolic radius rho in an n-dimensional hyperbolic space, regulate s times log(sinh(rho) / sqrt(n)) rather than rho itself, and use the same quantity to calibrate distance-logit temperature. This should make hyperbolic metric-learning behavior more invariant when embedding dimension, curvature, or model scale changes.
Regularize the Gram spectrum of selected neural layers so that its low-order moments match the spectral moments generated by a truncated q-boson Jacobi operator. Unlike a simple Frobenius or spectral-norm penalty, this controls several parts of the singular-value distribution simultaneously and can discourage harmful spectral tails without forcing all singular values to be equal.
Insert a short gKdV-inspired spectral flow between neural blocks to regularize rough feature maps without using an isotropic low-pass filter. The module applies a Fourier dispersive phase and derivative-coupled polynomial residual updates, with an optional finite factorial dilation penalty to encourage analytic-looking features.
Represent each token or graph node by an anti-Hermitian matrix latent state and replace a standard residual transformation with a discretized Lie-algebra vortex flow. The commutator nonlinearities are equivariant under global unitary conjugation, so the block can learn interactions without selecting a basis and preserves the anti-Hermitian state space when initialized there.
Construct a nested sequence of representations in which each coarser representation is obtained from the previous one by a 1-Lipschitz projection. Train prediction heads at multiple scales so coarse predictions remain stable and approximately recoverable from the finer representation, enabling early exit, token pooling, and controlled multiresolution inference.
Replace part of dense token mixing with a small bank of structured permutations acting on a hierarchical token tree. Diagonal inheritance shares the same local permutation across all descendant copies, while selector words activate one connector type at a chosen level and remain inactive on the next type, providing controllable multiscale receptive fields without constructing a dense attention matrix.
Construct p shared neural replicas of the same token or feature set, quotient their outputs by the cyclic group C_p, and train a power head to agree with the representation obtained from a jointly processed p-fold input. Add a filtration score whose value is nondecreasing under the power map and strictly increases on deliberately nontrivial replica combinations. The experiment tests whether this algebraically structured consistency signal is better than ordinary pairwise augmentation…
Replace an unrestricted collection of nested dyadic attention windows on a 2D token grid by a sparse antichain: no selected window may contain another selected window. Use the paper's exponential occupancy guarantee to control how many attention blocks reuse the same token, and add a differentiable log-moment penalty during training when exact antichain selection is relaxed. The expected benefit is bounded peak KV reuse and more predictable sparse-attention cost without discarding multiscale…
Use the paper's marginally irrelevant RG flow to schedule communication between two neural feature streams. A fast stream, such as transformer attention, can interact with a slower or more persistent stream, such as an SSM or low-frequency convolutional branch, through a gate that decreases like \(1/(1+a y_0 \ell)\) instead of remaining fixed across depth or training time. A learnable initial amplitude preserves adaptability while the inverse-logarithmic envelope suppresses harmful long-range…
Use the resolvent trace as a differentiable statistic that controls how strongly a learned routing or recurrent transition matrix returns to short cycles. Penalizing this quantity suppresses accidental short feedback loops, while matching a target trace can impose a desired memory profile in recurrent, graph, or mixture-of-experts architectures.
Insert a distribution-free rank warp before selected MLP or attention projections. For each scalar activation, replace its empirical rank u by the cumulative interval map induced by the Type-III derangetropy kernel, optionally followed by Gaussian or affine output calibration. The transform is invariant to strictly increasing reparameterizations of the feature and contracts the marginal toward central ranks, potentially reducing sensitivity to heavy tails and outliers.
Regularize the hidden-state trajectory of a sequence model so that the distance between states at positions i and j follows a controlled power-law profile in |i-j|. This explicitly prevents representation collapse over long contexts while avoiding the requirement that all distant states be maximally separated. Use alpha as a tunable geometry parameter and compare alpha against the effective hidden dimension using the paper's Euclidean realizability threshold.
Replace raw polynomial interactions between neighboring feature vectors with central polynomial interactions computed after subtracting the local feature mean. Keep separate second-, third-, and fourth-order channels and apply independent residual gates to them, so a uniform shift of every feature in a neighborhood cannot create artificial cross-order responses. This is a drop-in higher-order mixer for a small transformer or graph neural network.
Replace a collection of dense task-specific linear layers with a common sparse structural matrix and task-specific edge strengths. All tasks share the same learned connectivity pattern, but retain independent values on active connections, allowing parameter sharing without forcing identical interactions.
Replace arbitrary learned thresholds in a binary MoE or hierarchical latent router with a threshold at the batch mean of a learned scalar projection. Add a penalty when the entropy of either routed subgroup falls too far below the parent entropy, using the paper's sharp constant as the target. This discourages routing branches from becoming nearly deterministic or semantically impoverished while retaining a simple, cheap gating operation.
Implement \(\log(1+u)\) through the bounded residual \(\rho(u)\), refining the argument by repeated square roots and propagating the residual backward with a rational recurrence. Use this table-free approximation in softplus and log-sum-exp kernels, where millions of logarithms are evaluated and predictable arithmetic can outperform general-purpose library calls on vectorized hardware.
Treat the maximum dependency distance faithfully modeled by a finite neural architecture as an emergent correlation length, and estimate how it grows with depth, state size, or attention span. Fit the exponent \(\kappa\) and use it as an architecture-selection signal: a model with larger \(\kappa\) should acquire long-range competence more efficiently at equal parameter or FLOP budget.