Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

△ Mechanism confirmed, baseline not beaten 2026

Irregular-Domain Fourier Convolution

Use the truncated Fourier representation of an irregular domain as a reusable spectral mask inside an FFT convolution layer. This gives a cheap alternative to point-cloud neighborhood aggregation while explicitly suppressing contributions from outside the physical domain and improving behavior near corners, cusps, and holes.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: "Truncated Fourier Filtering" method for fast and high-order evaluation of integrals and convolutions in general domains arXiv:2608.25264
✗ Mechanism failed 2026

Long-Cycle Topological Graph Pooling

Construct a sparse radius graph over input samples or learned node embeddings, compute its cycle space, and remove the subspace generated by sufficiently short cycles. Feed the remaining quotient-cycle coordinates or Betti-rank estimate to a graph neural network as a global topological feature, or use them to guide pooling so that local redundant loops are collapsed while global loops are retained. The paper predicts that the threshold L approximately equal to |log r| graph hops is the critical…

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Detection of first homology via random geometric graphs in the thermodynamic regime arXiv:2608.25065
✗ Mechanism failed 2026

Canonical Tropical Segment State

Build a sequence or graph module whose state is a canonical set of affine tropical pieces rather than an opaque hidden vector. Compose consecutive segments by Minkowski addition of their lifted supports, merge alternatives by union followed by lower-hull reduction, and evaluate the resulting piecewise-linear function with a minimum.

Useful7/10
Difficulty7/10
Novelty8/10
Paper: On the Representational Geometry of Dynamic Programs arXiv:2608.25034
✗ Mechanism failed 2026

Neural Matrix Operator Inverse Head

Train a network to predict the context-dependent observation matrix rather than the latent inverse parameters themselves, then compute the latent parameters with a differentiable ridge-regression solve. This gives one model that can assimilate arbitrary observation vectors, exposes the conditioning of the inverse problem, and avoids forcing an MLP to learn the entire map from observations to parameters.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Real-time inverse solutions via neural matrix operators arXiv:2608.24833
Audited (legacy) 2026

Spectral Budgeted Embeddings

Replace uniform embedding dimensions with a globally budgeted allocation based on the estimated spectral complexity of each categorical feature. Tables whose category representations have large leading singular-value energy receive more dimensions, while high-cardinality tables are penalized because each extra dimension consumes more parameters.

Useful7/10
Difficulty4/10
Novelty6/10
Paper: Optimal Allocation of Embedding Dimensions under Finite-Sample Constraints arXiv:2608.24592
✗ Mechanism failed 2026

Tree-Structured Exact Expert Router

Replace independent top-k expert selection with a decision-diagram router that optimizes a quadratic surrogate over binary expert activations and continuous assignment weights. When expert redundancy or communication costs form a tree, partial routing decisions are merged whenever they have the same separator state, turning exponentially many candidate masks into a dynamic program over a small number of graph cuts.

Useful7/10
Difficulty7/10
Novelty8/10
Paper: Convexification of mixed-integer quadratic optimization via decision diagrams arXiv:2608.22815
Audited (legacy) 2026

Joint-Particle Distributional Critic

Replace independent per-action distributional value heads with a critic whose shared latent particle produces a vector of return samples for all actions simultaneously. Train the predicted joint return vector against a Bellman target vector formed from coupled counterfactual reward-transition samples, using a sliced Wasserstein loss. The greedy action is selected by the mean of the corresponding marginal particles, while shared particles retain cross-action dependence for learning and…

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Learning to Control Coupled-Dynamics Environments with Joint Markov Decision Processes arXiv:2608.22765
✗ Mechanism failed 2026

Sparse symbolic vector-field layer

Replace an opaque MLP vector field with a stack of trainable symbolic primitives that can express linear terms, monomials, products, and related analytic operations. Apply an L1 penalty and prune small primitive coefficients after rollout training, yielding a compact dynamics module that is cheaper to evaluate and easier to inspect.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Symbolic Neural ODEs: Learning interpretable models from time-series data arXiv:2608.22112
✗ Mechanism failed 2026

Delay-Ring Replay Memory

Replace a directed sequence-memory chain with a circular recurrent state propagated by a learned delayed convolution. The same learned kernel can support forward and reverse replay because replay direction is a dynamical mode of the ring, rather than requiring plasticity to explicitly learn both forward and backward synapses.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: Forward and reverse delay-driven hippocampal replay without symmetric plasticity arXiv:2608.21814
Unverified 2026

Delay-signature coincidence branches

Add a bank of temporal branches whose synapses apply heterogeneous delays before coincidence pooling. A branch selectively responds to an ordered pair or short sequence when the delay difference compensates the sequence interval, while low delay dispersion makes it primarily an order-invariant event detector. The branch threshold creates sparse, interpretable activations that can replace part of a recurrent or attention-based temporal module.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Axonal delay dispersion decides whether a neuron detects an event or a sequence, and predicts cortical column diameter arXiv:2609.04195
Unverified 2026

Lipschitz-Perturbation Brenier Latent Adapter

Replace an unconstrained latent-to-data map in a small generative model with a learned Brenier map T equal to the gradient of a convex potential, transporting a Gaussian latent distribution toward a density proportional to exp(-B) times the Gaussian density. Constrain B to be globally L-Lipschitz and use the resulting dimension-free Jacobian ceiling as an adaptive stability target. This should reduce pathological local expansion without requiring B itself to be convex or its Hessian to be…

Useful6/10
Difficulty7/10
Novelty6/10
Paper: Caffarelli Estimates under Lipschitz Perturbations arXiv:2609.04052
Unverified 2026

Asynchronous p-Mean Token Mixer

Replace dense token attention with K asynchronous local nonlinear consensus updates on a sparse token graph. Each selected token is moved to the unique incident p-mean of its neighbors, producing a tunable message-passing operator that is averaging for p=2 and nonlinear for other p.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Consensus time for asynchronous $\ell^p$ relaxation: graph dependence arXiv:2609.03856
Unverified 2026

Weighted Tree Dirichlet-to-Neumann Layer

Replace dense long-range token interactions with sparse message passing on a balanced hierarchical tree. Boundary token features are extended to internal tree nodes by weighted harmonic relaxation, and the resulting weighted normal derivative is used as a learned multiscale interaction or regularization signal.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Extension Problems for the Vladimirov--Taibleson Operator and the Hierarchical Laplacian arXiv:2609.03647
Unverified 2026

Probability-coordinate robust likelihood head

Replace a Gaussian regression likelihood with a chart-pulled exponential-family head. The network predicts a natural parameter eta, while each target y is mapped to a bounded probability coordinate u=G(y). Because the sufficient statistic T(u) is bounded, extreme targets cannot create arbitrarily large parameter gradients. The chart determines the tail class, while eta controls the distribution in probability space.

Useful6/10
Difficulty4/10
Novelty6/10
Paper: G-Exponential Families through Probability Coordinates arXiv:2609.03208
Unverified 2026

Nonreciprocal mutant feature fields

Replace a single residual feature field with two nonnegative streams: a primary representation A and a mutant or corrective representation B. A produces B through a one-way source term, while B suppresses A, creating an explicitly tunable competition mechanism that can prevent feature collapse and encourage complementary representations.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Death by mutants: unusual multicritical dynamics in a two-species model for absorbing state transitions arXiv:2609.03123
Unverified 2026

Low-Rank Gromov-Wasserstein Cross-Attention

Replace ordinary feature-only cross-attention between two graph sets with a transport coupling optimized using pairwise structural costs. Use the paper's positive-semidefinite graph-kernel factorization to compute the quadratic GW interaction without materializing an O(N_0^2 N_1^2) tensor, and use the inexact-gradient convergence bound to control truncated or approximate updates. The resulting coupling can be used directly as attention weights or as a soft graph-matching matrix.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Discrete Gromov-Wasserstein Duality: Algorithms and Isomorphism Testing arXiv:2609.03094
Unverified 2026

Manifold-Adaptive Positive Quadrature Attention

Replace dense aggregation over patches, points, or graph neighbors with a state-dependent positive quadrature rule. A latent encoder produces the current manifold coordinate z, a small controller chooses a subset of entities, and a convex redistribution step assigns nonnegative weights so that the subset preserves the full aggregation on a learned feature basis. The method is especially suitable for neural operators and vision attention where many spatial entities are redundant along the data…

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Dimensional hyperreduction of nonlinear finite element models via empirical cubature with manifold-adaptive weights arXiv:2609.03068
Unverified 2026

Fermi-Dirac Capacity Projection for MoE Routing

Replace heuristic softmax load balancing with a convex entropic projection that computes expert assignment fractions satisfying explicit linear capacity and load constraints. The projection is bounded componentwise and keeps poorly identified routing decisions away from extreme 0/1 allocations, reducing expert collapse and overload under noisy or shifted batches.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: An Entropic Factor Model for Robust Portfolio Replication arXiv:2609.03552
Unverified 2026

Exact hierarchical polar differential layer

Replace a Cartesian grid or singular polar convolution near the center of a disk with a hierarchy of polar-spline coefficient features. Use separate coefficient spaces for scalar fields, vector 1-forms, and 2-forms, together with sparse derivative matrices satisfying the discrete complex identity D1 D0 = 0. Refine only cells with large learned or physics residuals, while retaining the pole constraints that make the representation smooth despite the collapsed angular boundary.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Adaptively-refinable polar-spline discrete differential forms: hierarchical construction, exactness, and applications arXiv:2609.03461
Unverified 2026

Balanced Laguerre Codebook Layer

Replace ordinary nearest-neighbor or softmax codebook assignment with a capacity-constrained Laguerre assignment whose cells have prescribed masses. Optimize the codebook using the semi-discrete quadratic Wasserstein energy, whose gradient moves every site toward the barycenter of its balanced cell. This directly prevents prototype collapse and gives explicit control over expert or codeword utilization.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Semi-discrete quadratic Wasserstein energy and state-dependent Langevin exploration arXiv:2609.03405
Unverified 2026

Sector-Parametrized Neural Operator

Replace a recurrent, SSM, or residual linear operator W by an exact affine image of a contractive matrix sigma. The resulting W lies in a prescribed matrix ball and, for the positive-real choice of parameters, satisfies a sector inequality such as W+W^* >= 0. This gives an anisotropic, learnable alternative to plain spectral normalization: the allowable operator set is convex and its center and left/right radii can encode known directional scales.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Hyperpositive functions, sector bounded functions and a trace formula arXiv:2609.03403
Unverified 2026

Resolution-Invariant Hilbert-Space CNF Prior

Replace a finite-dimensional latent prior for fields or trajectories with a continuous flow acting on function coefficients in a separable Hilbert space. A Gaussian reference with an explicit covariance spectrum is transformed by a learned vector field, so the model can represent smooth, non-isotropic function distributions while sharing parameters across discretization resolutions.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Learning Informative Prior with Infinite-Dimensional Continuous Normalizing Flow for Bayesian Inverse Problem arXiv:2609.03343
Unverified 2026

Uniform Species Architecture Sampler

Translate a recursive neural architecture grammar into a combinatorial specification and use its generating function to sample architectures uniformly conditional on size. This removes the severe shape and branching biases of naive random tree generation, producing a better-covered architecture-search population at the same evaluation budget.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Enumerating Chemical Structures, Stereoisomers, and Stereogenic Units with Analytic Combinatorics arXiv:2609.02932
Unverified 2026

Copula Correction Layer for Joint Predictive Calibration

Add a post-processing or differentiable output layer that first matches each predicted marginal distribution to the data marginal and then corrects the remaining joint dependence by transporting the predicted copula to the observed copula. This directly targets the discrepancy that sequential marginal or quantity-of-interest constraints cannot remove.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Copula Transformations for Data-Consistent Inversion arXiv:2609.02832