Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

△ Mechanism confirmed, baseline not beaten 2026

Signed spectral attention

Replace a quadratic pairwise attention or graph aggregation kernel with a compact, translation-invariant indefinite kernel approximated by signed random Fourier features. The feature map preserves the kernel's negative spectral mass through a diagonal sign matrix, so the resulting linear-time aggregation can represent similarities that ordinary positive-definite random features cannot.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Signed random Fourier features for fast density estimation with indefinite kernels arXiv:2608.29265
✗ Failed on benchmark 2026

Topological Fluctuation Graph Layer

Replace a deterministic graph propagation layer by a stable stochastic linearized latent dynamics whose frequency-resolved covariance matrix defines spectral bands. Train or initialize the graph operator so that a selected covariance band has a nonzero Chern number and remains separated by a measurable spectral gap, producing representations that are robust to local perturbations and can support boundary-localized responses.

Useful7/10
Difficulty7/10
Novelty8/10
Paper: Topology of Fluctuation Bands in Chiral Active Matter arXiv:2608.26055
△ Mechanism confirmed, baseline not beaten 2026

Block-TT 3D Neural Operator

Represent a large linear map acting on a Cartesian 3D grid and multiple physical channels as a TT-matrix, while retaining separate TT blocks for channel couplings that have different semantics. Apply the layer by sequential contractions with TT cores rather than materializing a dense matrix or a full 3D convolution kernel. Rank truncation provides an explicit accuracy-versus-memory knob and can be applied after optimizer updates.

Useful7/10
Difficulty5/10
Novelty5/10
Paper: Tensor-Train Methods for 3D Linear Elasticity: Block and Global Operator Representations with Solver Performance Analysis arXiv:2608.23595
✓✓ Beats tuned baseline 2026

Gated Local-Global Graph Attention

Replace dense graph self-attention with two parallel branches: exact softmax attention only over graph neighbors and a global linear-attention branch that summarizes all nodes through feature-space statistics. A learned node-wise gate interpolates between the branches, allowing locally structured nodes to use sparse attention while retaining a global-information path.

Useful7/10
Difficulty4/10
Novelty5/10
Paper: Graph-Native Attention Acceleration for Attack Detection in Cyber-Physical Systems arXiv:2608.23414
△ Mechanism confirmed, baseline not beaten 2026

Spectral Cross-Block Averaging Layer

Construct a cheap graph or token-mixing operator by partitioning nodes into k blocks using the bottom nonconstant eigenvectors of P squared, then replacing dense pairwise mixing with conditional averaging inside each block followed by one baseline propagation step. Unlike ordinary spectral clustering, the bottom modes target partitions where block labels are rapidly destroyed by P, producing an aggressively mixing representation layer rather than a community-preserving pooling layer. The…

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Spectral partitioning for $k$-block averaging kernels of finite Markov chains arXiv:2608.21466
△ Mechanism confirmed, baseline not beaten 2026

T-Robinson Hierarchical Attention

Replace flat all-pairs attention with attention neighborhoods induced by a compatible tree over tokens, patches, nodes, or retrieved items. Retain exact or approximate attention inside nearby tree subtrees and add a path-monotonicity regularizer so semantically distant endpoints are not more similar than intervening tree neighbors.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: T-Robinson Spaces: Structure, Recognition, and Applications to Real Data arXiv:2608.21248
△ Mechanism confirmed, baseline not beaten 2026

Resistance-certified tree attention

Replace an arbitrary graph-attention mask with a fractional edge mask lying in the intersection of the spanning-tree polytope and twice the matching polytope. The mask represents a distribution over connected spanning trees while imposing expected degree at most two at every vertex, after which sampled trees can be used for sparse message passing.

Useful7/10
Difficulty7/10
Novelty7/10
Paper: Resistance Curvature: Recognition, Polyhedral Structure, and Graph Products arXiv:2608.20778
✗ Failed on benchmark 2026

Shared-response expert ranking

Build a label-free router for a finite library of neural operators by estimating one shared physical target response from an anchor prediction and using it to rank every candidate through inner products with candidate differences. The method avoids running a full residual-based diagnostic independently for every expert and can be used either to select the best expert or to form a corrected weighted combination.

Useful7/10
Difficulty6/10
Novelty8/10
Paper: Shared Physics Responses Recover Hidden Rankings in Neural Operator Libraries arXiv:2608.20441
✓✓ Beats tuned baseline 2026

PPMI-Gated BCM Sparse Graph Encoder

Replace a dense graph embedding table or end-to-end GNN encoder with a fixed-width binary SDR learned from streaming random-walk context pairs. Use PPMI to amplify informative node-context pairs and a local BCM update to learn detector columns, followed by k-winner-take-all binarization. The resulting sparse code can be used directly for node classification, link prediction, retrieval, or as input to a small downstream predictor.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Columnar-Embedder: A Biologically Inspired Cortical Architecture for Binary Sparse Distributed Graph Representations arXiv:2608.20408
✗ Failed on benchmark 2026

Equation-addressable equilibrium layer

Build a recurrent or implicit neural layer from a bipartite graph containing variable nodes and equation or mechanism nodes, rather than a directed graph containing only variables. The forward pass solves all mechanism residuals simultaneously, while an intervention replaces one selected equation and fixes its target variable; this distinguishes interventions that impose the same value through different mechanisms. The resulting module is suitable for equilibrium world models, differentiable…

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Causal Reasoning with Bipartite Graphical Causal Models arXiv:2608.19831
✓✓ Beats tuned baseline 2026

Exact Moment Message Passing

Replace per-particle message evaluation in a point-cloud or particle-based neural layer with exact box moments. Particles inside a box are compressed into a fixed tensor of monomial sums, and every query in that box evaluates the same piecewise-polynomial interaction from those moments, reducing work from particle-query pairs to particles plus occupied boxes.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Exact hierarchical algorithms for accelerating particle--mesh coupling in sparse-grid particle-in-cell methods arXiv:2608.19702
△ Mechanism confirmed, baseline not beaten 2026

Harmonic-Mode Branch for Topological Memory

Do not force Hodge dissipation onto harmonic edge modes, because these modes are precisely the obstruction to global coercivity. Split the latent state into dissipative coexact modes and a finite-dimensional harmonic branch, and use harmonic-decoupled interactions so each harmonic coordinate defines an invariant affine fibre with its own attractor.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Hodge Coercivity and Global Dynamics in Two-Field Edge-Cochain Systems with MHD-Type Cancellation arXiv:2608.19360
△ Mechanism confirmed, baseline not beaten 2026

Ordered Diffusion Message Passing

Use a learned scalar ordering function to turn a symmetric local Gaussian graph kernel into a directed, row-stochastic message-passing operator. The asymmetric tilt lets neighboring nodes communicate preferentially along an inferred dynamical direction, while the Gaussian factor retains locality and diffusion-like smoothing.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Ordered Diffusion Kernels arXiv:2608.18019
✗ Failed on benchmark 2026

Gauge-Covariant Wilson-Loop Regularization

Attach an SU(2) transport matrix to every directed edge of a graph neural network and penalize nontrivial plaquette holonomies instead of penalizing individual edge transformations. The regularizer is invariant to arbitrary local changes of latent representation frame, encouraging path-consistent relational features without requiring all edges to share one global coordinate system.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Frustration without Glass: A Non-Abelian Gauge Model of Network Compatibility arXiv:2608.17817
✗ Mechanism failed 2026

Exact-Curl Neural Field Output

Make a neural network predict a vector potential rather than a magnetic or velocity field, then obtain the physical vector field with a fixed differentiable discrete curl. The reconstructed field satisfies the discrete divergence-free constraint exactly, eliminating divergence-penalty tuning and preventing constraint drift during long rollouts.

Useful7/10
Difficulty4/10
Novelty5/10
Paper: A Structure- and Pressure-Positivity-Preserving Semi-implicit IMEX Finite Volume Scheme for Ideal MHD at All Acoustic Mach and Alfvén Mach Numbers with Generic Equation of State arXiv:2608.15837
△ Mechanism confirmed, baseline not beaten 2026

Weighted Resolvent-Equivariant Attention

Add a weighted reflection symmetry to an attention or graph-propagation matrix instead of requiring ordinary permutation equivariance. For paired positions or graph nodes related by an involution, penalize the failure of the propagation operator to commute with the weighted reflection; this makes all geometric multi-step propagations symmetry-compatible. The method is suitable for data with mirror, reversal, paired-agent, or left/right structure where the two sides have unequal importance…

Useful7/10
Difficulty4/10
Novelty6/10
Paper: Resolvent intertwining and spectral duality in Markov chains with geometric resetting arXiv:2608.15140
✗ Failed on benchmark 2026

Gauge-Free Spectral OT Layer

Parameterize an entropic OT cost only in directions that can change the transport plan, removing row-plus-column potential directions that are invisible because of OT gauge invariance. Whiten the remaining feature coordinates using their empirical covariance, producing an OT layer whose identifiable parameters have substantially more uniform sensitivity.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Sinkhorn Linearization and the Spectral Proxy: Unifying the Statistical and Algorithmic Theory of Feature-Parameterized Inverse Optimal Transport via a Single Spectral Sandwich arXiv:2608.13201
✓✓ Beats tuned baseline 2026

Displacement-Huber distribution pooling

Replace ordinary Wasserstein or arithmetic pooling of distribution-valued features with a barycenter whose individual quantile displacements are Huberized. Small changes between input distributions remain averaged quadratically, while a corrupted token, expert, graph neighborhood, or augmentation cannot move the pooled distribution arbitrarily far. The module is especially cheap for one-dimensional distributions represented by fixed quantile vectors.

Useful7/10
Difficulty4/10
Novelty7/10
Paper: Huber-Wasserstein barycenters for robust distribution-valued data arXiv:2608.13131
△ Mechanism confirmed, baseline not beaten 2026

Measurement-Space Neural Operator with Mesh Transfer

Build a neural operator around explicit input and output measurement spaces rather than forcing the network to consume and emit a fixed grid. The same learned latent surrogate can be reused on alternative sensor layouts or query meshes through reconstruction and re-encoding maps, with a consistency loss enforcing agreement between measurement pipelines.

Useful7/10
Difficulty6/10
Novelty5/10
Paper: Kernel Methods for Learning Operators with Multiple Inputs and Outputs arXiv:2608.11831
✗ Mechanism failed 2026

Regret-aware evidential cost compression

Attach an evidential cost head to a neural graph model, representing each edge cost by a weighted set of interval boxes, and compress this representation before the downstream shortest-path or routing solver. Instead of minimizing Jaccard or Jousselme distance between the original and compressed mass functions, choose merges that minimize the induced cost error on the currently selected route, while enforcing a conservative monotonicity condition so that the resulting path regret is bounded.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: Decision-Aware Approximation of Belief Functions for Evidential Combinatorial Optimization arXiv:2608.10650
✗ Failed on benchmark 2026

Følner-Gated Message Passing

Add an online receptive-field expansion monitor to a graph neural network and use it to gate message-passing depth or invoke graph pooling. For a sampled node set F and propagation neighborhood K, continue fine-scale propagation only while the growth ratio |KF|/|F| is close to one; when it is persistently expansive, replace further propagation with pooling, local attention, or long-range skip messages. This transfers the paper's Følner-versus-paradoxical mechanism into an architecture-level…

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Fiberwise amenability of étale groupoids arXiv:2608.09796
△ Mechanism confirmed, baseline not beaten 2026

GECC-Gated Loop-Aware Message Passing

Construct order-n generalized edges from intersections of local ego-subgraphs and use their overlap statistics to correct ordinary one-hop aggregation. A learned gate should activate the correction only when local generalized-edge closure is high, because dense but internally inconsistent overlaps are precisely where naive loop corrections can become unreliable.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Ensemble-level loopy message passing with generalized-edge closure for percolation arXiv:2608.09397
✗ Failed on benchmark 2026

Differentiable Persistence Landscape Layer

Convert each persistence diagram produced from an input, intermediate feature map, or graph filtration into a discretized persistence landscape and feed it to an MLP or concatenate it with ordinary neural features. Unlike a variable-size list of birth-death pairs, the landscape has a fixed tensor shape and is provably nonexpansive with respect to the diagram Wasserstein distance.

Useful7/10
Difficulty4/10
Novelty5/10
Paper: A Hilbert space embedding of persistence diagrams and barcodes arXiv:2608.08858
△ Mechanism confirmed, baseline not beaten 2026

Hodge-Selective Edge Dynamics

Replace an unconstrained edge-feature residual update in a graph neural network with separate cut-space and harmonic-space updates. The cut branch carries transfer information visible at nodes, while the harmonic branch models cycle circulation and can be given an independently chosen contraction rate, preventing persistent or unstable circulation features from contaminating node predictions.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: From a Scalar Parabolic Oscillator to Topological Thermostats: Selective Feeback Control of Harmonic Flow Modes arXiv:2608.07768