Every idea extracted from recent arXiv mathematics papers — verified and unverified. Click an idea to open its full card; badges show the empirical verdict.
Replace an ordinary group-L1 penalty on structured neural components with a powered ratio-of-norms penalty applied to their nonnegative importance magnitudes. The ratio encourages importance to concentrate on a small number of heads, channels, or experts while being less sensitive to arbitrary rescaling of the underlying weights. After training, components with small importance can be physically removed and the model can be fine-tuned.
Add a local Jacobian spectral regularizer and an initialization sweep to steer a looped transformer away from uncontrolled near-unit dynamics. The goal is to prevent examples from entering a fold-critical regime with very long relaxation times, or alternatively to deliberately target a controlled critical regime when adaptive test-time compute is useful.
Replace an unconstrained relative-position attention-bias table by a learnable two-dimensional Fourier representation, or regularize the bias toward a small Fourier ratio. The ratio favors coefficient concentration without depending on the absolute scale of the bias, allowing the trained bias to be reconstructed from a small number of dominant frequencies.