Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✗ Mechanism failed 2026

Signed-Distance Ambient FNO

Embed every variable-domain sample into one fixed ambient hyperrectangle and append its signed-distance function as an additional channel to the operator input. Deterministically extend fields outside the physical domain, resample them onto a shared latent grid, apply standard Fourier layers, then interpolate and mask the output on the requested target discretization. The network learns the operator rather than a separate geometry encoder, so the same weights can be used across shapes and mesh…

Useful7/10
Difficulty4/10
Novelty6/10
Paper: ADEx-FNO: A Unified Ambient-Domain Framework for Fourier Neural Operators on Varying Geometries arXiv:2608.08608
△ Mechanism confirmed, baseline not beaten 2026

Tolerance-controlled adaptive low-rank layers

Replace selected dense neural-network operators by low-rank factors whose rank is selected by a randomized residual test at a user-specified tolerance. Construct candidate bases in large blocks for efficient matrix operations, then prune the block to the smallest rank that passes the residual criterion instead of treating the block size as the final rank.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Adaptive, Matrix-Free Low-Rank Approximation arXiv:2607.06758
Unverified 2026

TT Cubic Sign Router

Replace explicit thresholding of a large structured routing-logit tensor with the paper's cubic sign iteration applied directly to its TT cores. The resulting approximate sign field can be converted into binary expert gates or binary activation masks without materializing all tensor entries, enabling routing over exponentially large Cartesian index spaces. Use TT rounding after each Hadamard product to prevent rank explosion, and compare the method against dense thresholding on smaller…

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Iterative tensor network transformations for element-wise evaluation of elementary and filtering functions arXiv:2608.17135
Unverified 2026

Orientation-consistent adaptive tetrahedral neural field

Represent a 3D neural field using high-order polynomial coefficients attached to an adaptively refined tetrahedral mesh, with a small MLP predicting residual corrections from local coordinates. Refine only tetrahedra whose prediction, rendering, or PDE residual is large, and use globally ordered vertices so neighboring tetrahedra share identical face and edge coefficients without hanging-node constraint solves.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: An $h$-adaptive Tetrahedral Spectral Element Method with Applications to Kohn-Sham Density Functional Theory arXiv:2608.14006