Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

2 ideas found

Category: Architecture Solves: Regularization ML: Regularization Clear all

Unverified 2026

Weighted Tree Dirichlet-to-Neumann Layer

Replace dense long-range token interactions with sparse message passing on a balanced hierarchical tree. Boundary token features are extended to internal tree nodes by weighted harmonic relaxation, and the resulting weighted normal derivative is used as a learned multiscale interaction or regularization signal.

Useful6/10

Difficulty5/10

Novelty6/10

Architecture Accuracy Scalability Regularization

Paper: Extension Problems for the Vladimirov--Taibleson Operator and the Hierarchical Laplacian arXiv:2609.03647

Unverified 2026

Anisotropic Anharmonic Diffusion Layer

Insert a learnable semigroup layer that evolves features according to a positive operator combining frequency damping and spatially varying confinement. Unlike isotropic Gaussian smoothing, the layer can damp selected frequencies differently along different axes and can suppress activations in learned spatial regions, while the positive-semigroup construction prevents amplification.

Useful6/10

Difficulty5/10

Novelty7/10

Architecture Stability Accuracy Regularization

Paper: Sharp Time-Decay Estimates for Fractional Heat Semigroups Associated with Polynomial Anharmonic Oscillators arXiv:2607.17580