Every idea extracted from recent arXiv mathematics papers — verified and unverified. Click an idea to open its full card; badges show the empirical verdict.
Replace dense self-attention by a sparse attention graph whose neighborhoods satisfy the paper's size-dependent expansion condition. This preserves a logarithmically controlled route for every token subset to communicate with the rest of the sequence, reducing quadratic attention cost without allowing disconnected or poorly mixed token groups.
Replace dense token-to-token attention by a multiscale spiderweb communication pattern. Tokens first aggregate upward through a dyadic hierarchy, communicate horizontally only with a small number of cells at the appropriate height, and then receive information broadcast downward. Hyperbolic distance supplies a principled rule for choosing the height at which two tokens interact: nearby tokens interact at fine scales, while far-apart tokens interact through coarse representatives.
Replace dense pairwise interactions between all forecast horizons with nested time-shell summaries. For sorted horizons, the readout at shell j receives a cumulative embedding of all coefficients or queries assigned to later horizons, reproducing the paper's dependence on products such as \(\Pi_j=\prod_{l>j}e^{\alpha_l}=e^{\sum_{l>j}\alpha_l}\). This gives an \(O(Kd)\) multi-horizon interaction instead of an \(O(K^2d)\) temporal attention block and should work best for weak-memory…
Replace a recurrent token-mixing operation with a finite-state carrier scan over binary or quantized token features. The local transition table is constrained to conserve a scalar token weight and to be bijective, making the mixer reversible, constant-memory, and less prone to activation drift than a generic recurrent layer.
Use an approximate decision diagram to select a structured subset of neurons, channels, attention heads, or attention edges when their quadratic interactions are sparse or inverse-sparse. Merge states that agree on a local interaction boundary and accept a tunable epsilon loss in the pruning objective, obtaining a representation whose size is linear in model width for fixed accuracy tolerance.