Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

Unverified 2026

Anisotropic Anharmonic Diffusion Layer

Insert a learnable semigroup layer that evolves features according to a positive operator combining frequency damping and spatially varying confinement. Unlike isotropic Gaussian smoothing, the layer can damp selected frequencies differently along different axes and can suppress activations in learned spatial regions, while the positive-semigroup construction prevents amplification.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Sharp Time-Decay Estimates for Fractional Heat Semigroups Associated with Polynomial Anharmonic Oscillators arXiv:2607.17580