✓✓ Beats tuned baseline
2026
Replace a dense mixing or attention matrix on tokens arranged on a Cartesian grid by a product of learned or fixed one-dimensional concentration operators. The layer applies one axis operator at a time, reducing parameter and compute cost while enforcing that the global operator is a positive contraction with controlled spectral leakage.
Useful7/10
Difficulty5/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Replace dense attention between tokens on opposite sides of a one-dimensional boundary or segment split with a dyadic low-rank approximation of a Cauchy/Hankel distance kernel. Each distance-scale block uses O(log(1/\varepsilon)) features, and the number of active scales grows only logarithmically with context length after discarding a narrow boundary layer. This is especially suitable for a relative-position attention branch or state-space-like long-range branch, rather than arbitrary…
Useful7/10
Difficulty5/10
Novelty6/10
✗ Failed on benchmark
2026
Construct a spatiotemporal neural block from localized functions of a learned parabolic operator instead of unrestricted attention or convolution. Use one filter for fine-scale diffusion and another for coarse-scale temporal aggregation, with the scale ratio controlling information propagation. The block should suppress distant interactions while still permitting long-range mixing through coarse filters.
Useful7/10
Difficulty6/10
Novelty6/10
✗ Failed on benchmark
2026
Use separate measurement-domain and target-domain token sets so a network can infer a field on one spatial domain from sparse observations on another in one forward pass. The same decoder can answer arbitrary target query points, avoiding an optimization loop for each inverse instance.
Useful7/10
Difficulty5/10
Novelty6/10
✗ Mechanism failed
2026
Replace a dense coordinate-kernel interaction among N points by an orthogonal samplet transform with a sparse detail-detail matrix and a small polynomial branch. Detail basis vectors have vanishing moments, so smooth low-frequency behavior is represented by a few polynomial coefficients while localized residual interactions become sparse in the transformed domain.
Useful7/10
Difficulty6/10
Novelty7/10
Unverified
2026
Add a consistency loss for a neural model processing irregularly sampled functions or sequences, comparing its latent covariance representation before and after a time-coordinate warp. The target discrepancy should be near zero for affine transformations but should grow with the normalized curvature, bandwidth, and local sparsity of a nonlinear warp.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Attach each token or graph node a learned scalar charge q_i and add a fractional stable kernel K_ij = exp(-tau D |q_i-q_j|^alpha) to the interaction mechanism. Constrain 0 < alpha <= 2, the exact range in which the kernel is positive semidefinite for arbitrary finite real charge sets, and optionally make tau layer-dependent to obtain multiscale interactions. This provides a principled alternative to unconstrained learned distance biases and can be used either as an attention-logit bias or as a…
Useful6/10
Difficulty4/10
Novelty5/10
Unverified
2026
Replace a global attention truncation rule with a per-query local-tail budget. For each query token, retain nearby or high-priority keys until the estimated discarded interaction strength is below a target epsilon; this uses the paper's central distinction between local tail mass and the extensive norm of the discarded operator. The resulting attention pattern can allocate long-range computation only to tokens whose local tail is large.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Generate memory slots or attention keys by applying learned unitary transformations to one seed vector instead of storing every slot independently. Select transformations sequentially when they add a sufficiently new direction and reject phase-equivalent or highly coherent candidates, targeting a well-conditioned near-Parseval frame.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Add a multiscale penalty to transformer token-mixing activations when they are simultaneously concentrated on a spatial or token subset and on a separated, irregular frequency subset. The penalty uses the fractal uncertainty scaling law to discourage hidden states from collapsing onto narrow token patterns and narrow spectral bands, potentially improving robustness to token masking and frequency-corrupted inputs.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace a freely learned finite impulse-response mixing kernel with a matrix polynomial whose roots are constrained to the unit circle. The resulting block-Toeplitz operator has an explicitly positive semidefinite spectral construction, while increasing the polynomial degree gives a systematic capacity knob for approximating matrix-valued frequency responses.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace generic cross-scale mixing with a fixed-shape or lightly parameterized Erdelyi-Kober fractional convolution over logarithmic scale. The fractional order controls how strongly nearby scales are emphasized, while the exponential tail parameter controls the receptive field over distant scales, providing an interpretable alternative to dense cross-scale attention.
Useful6/10
Difficulty4/10
Novelty8/10
Unverified
2026
Represent each Euclidean input point by its responses to randomly sampled half-spaces, then center those responses by the minibatch or source-distribution half-space occupancy. Use squared distances between these representations as a geometry-preserving substitute for raw Euclidean distances in graph-NN edge construction, cross-domain retrieval, or geometry-aware attention. The layer can compare point clouds from different ambient dimensions because each domain has its own half-space dictionary…
Useful6/10
Difficulty4/10
Novelty6/10
Unverified
2026
Regularize a learned self-adjoint contraction so that its eigenvalues move toward 0 or 1 rather than accumulating in the transition interval. This suppresses ambiguous mixing modes and can enable a smaller binary spectral approximation at inference.
Useful6/10
Difficulty4/10
Novelty6/10
Unverified
2026
Replace a collection of overlapping sliding-window features with approximately orthogonal incremental features: the length-m feature contains information not predictable from shorter consecutive windows. Use the paper's transition-operator Toeplitz precision matrix to decorrelate the resulting sequence of window features before attention, suppressing duplicated local evidence and improving conditioning.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Use a density-weighted kernel operator whose features are multiplied by sqrt(rho) on both input and output sides. Under bounded kernels, changing the density by L1 distance changes the operator in Hilbert-Schmidt norm only at square-root order, giving a directly testable robustness guarantee for adaptive attention or graph layers.
Useful6/10
Difficulty3/10
Novelty6/10
Unverified
2026
Convert a density-dependent operator acting only on active coordinates into an ambient neural layer by embedding with sqrt(rho) and setting its action to zero on the zero-density defect. This produces the minimum-norm extension and prevents arbitrary or unstable outputs on coordinates that the current density declares unobservable.
Useful6/10
Difficulty4/10
Novelty7/10
Unverified
2026
Augment a causal sequence model with a small hierarchy of prefix summaries weighted by powers of the logarithmic rank of each preceding token. The summaries retain order-sensitive deviations from a baseline representation while costing O(KNd) for sequence length N, hierarchy width K, and hidden dimension d, instead of O(N^2d) dense attention.
Useful6/10
Difficulty4/10
Novelty6/10
✗ Failed on benchmark
2026
Replace or augment geometric attention on spatial or point-cloud tokens with a positive fractional kernel containing the paper's inverse-square origin factor. This gives tokens near a designated singular center a controlled increase in receptive-field influence while preserving a scale-invariant distance decay, which may help models represent cusp-like fields, radial singularities, and multiscale spatial interactions.
Useful6/10
Difficulty5/10
Novelty7/10
✗ Failed on benchmark
2026
Replace coordinate-wise mean pooling of metric-valued items with a finite representation of their free integral. Each item x in a pointed metric space M is represented through evaluations of learned Lipschitz probes, and the pooled feature is the weighted integral of those probe values. A dual Lipschitz critic estimates the free-space norm of differences between pooled groups, making the representation sensitive to metric geometry while remaining permutation-invariant.
Useful6/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Add a learned Riesz-transform branch that extracts normalized spatial gradients after diffusion by a positive parabolic operator. The diffusion branch carries smooth semantic content, while the Riesz branch represents boundaries, motion changes, and graph discontinuities. Resolvent smoothing makes the derivative branch less sensitive to feature noise than directly applying a finite difference.
Useful6/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Parameterize a relative-position or lag-decay function as a finite positive mixture of exponentials instead of learning arbitrary attention bias values. The resulting kernel is completely monotone on positive distances, so it is nonnegative, decreasing, and has alternating derivative signs; the mixture provides several learned memory scales without allowing oscillatory or unstable long-range biases.
Useful6/10
Difficulty4/10
Novelty7/10
✗ Mechanism failed
2026
Use the paper's q-ary overlap inequality as a regularizer for categorical neural networks. Two independently sampled attention, routing, or message-passing supports should rarely overlap in many locations; penalizing the moment q^{|S\cap S'|} discourages redundant histories and correlated interference between heads or experts.
Useful6/10
Difficulty3/10
Novelty6/10
✗ Mechanism failed
2026
Replace scalar entropy penalties on attention maps with a matrix-valued heat-flow regularizer over a circular or periodic token coordinate. Each position stores a positive semidefinite matrix describing coupled heads, experts, or channels; heat smoothing is constrained by the sharp modified log-Sobolev and Bogoliubov–Kubo–Mori contraction rather than an arbitrary smoothing coefficient. This should suppress high-frequency routing noise while preserving positive matrix structure and reducing…
Useful6/10
Difficulty6/10
Novelty7/10