Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

△ Mechanism confirmed, baseline not beaten 2026

Farkas-Certified Neural Safety Shield

Insert a constraint-reduction layer between a policy network and its executed action. The policy proposes an action, while the layer retains only geometrically extreme collision and obstacle constraints and verifies that every discarded halfspace is implied by the retained ones through nonnegative conic multipliers. The reduced projection or quadratic program is therefore equivalent to the full tightened safety filter whenever certification succeeds, but uses substantially fewer constraints.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Scalable Tube-Tightened Multi-Agent Safety via Certified Constraint Reduction arXiv:2608.25323
△ Mechanism confirmed, baseline not beaten 2026

Reverse-Sweep Backward for Block-Implicit Layers

Replace unrolled autodiff through an ordered block-implicit neural layer with a custom reverse sweep that solves one small transposed local system per forward block update. The backward computes the exact gradient of the executed finite-depth solver while avoiding a global Jacobian and retaining only compact block information.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: Differentiate the Solver, Not the Equation: Reverse-Sweep Adjoints for Block Implicit Simulation arXiv:2608.08559
✗ Mechanism failed 2026

Conditional-information-preserving pooling

Replace ordinary token merging or graph pooling with a learned block map whose output preserves information about a remote target conditioned on the surrounding coarse representation. The paper's majority-spin counterexample gives a concrete failure mode: two microscopic configurations mapped to the same pooled token can imply different predictions for distant variables.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Renormalization group and long-range conditional mutual information in hierarchical models arXiv:2609.02141
△ Mechanism confirmed, baseline not beaten 2026

Mean-field rainbow relation router

Replace independent edge-type logits in a relational graph neural network with a mean-field fixed-point router derived from a colored ERGM. Each edge's color distribution is influenced by its own relation bias and by the expected number of rainbow triangles it forms with neighboring edges, allowing the model to learn coordinated multilayer structures.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Colorful Exponential Random Graph Models arXiv:2608.31130
✓✓ Beats tuned baseline 2026

Gated Local-Global Graph Attention

Replace dense graph self-attention with two parallel branches: exact softmax attention only over graph neighbors and a global linear-attention branch that summarizes all nodes through feature-space statistics. A learned node-wise gate interpolates between the branches, allowing locally structured nodes to use sparse attention while retaining a global-information path.

Useful7/10
Difficulty4/10
Novelty5/10
Paper: Graph-Native Attention Acceleration for Attack Detection in Cyber-Physical Systems arXiv:2608.23414
△ Mechanism confirmed, baseline not beaten 2026

T-Robinson Hierarchical Attention

Replace flat all-pairs attention with attention neighborhoods induced by a compatible tree over tokens, patches, nodes, or retrieved items. Retain exact or approximate attention inside nearby tree subtrees and add a path-monotonicity regularizer so semantically distant endpoints are not more similar than intervening tree neighbors.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: T-Robinson Spaces: Structure, Recognition, and Applications to Real Data arXiv:2608.21248
✗ Mechanism failed 2026

Regret-aware evidential cost compression

Attach an evidential cost head to a neural graph model, representing each edge cost by a weighted set of interval boxes, and compress this representation before the downstream shortest-path or routing solver. Instead of minimizing Jaccard or Jousselme distance between the original and compressed mass functions, choose merges that minimize the induced cost error on the currently selected route, while enforcing a conservative monotonicity condition so that the resulting path regret is bounded.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: Decision-Aware Approximation of Belief Functions for Evidential Combinatorial Optimization arXiv:2608.10650
✗ Failed on benchmark 2026

Følner-Gated Message Passing

Add an online receptive-field expansion monitor to a graph neural network and use it to gate message-passing depth or invoke graph pooling. For a sampled node set F and propagation neighborhood K, continue fine-scale propagation only while the growth ratio |KF|/|F| is close to one; when it is persistently expansive, replace further propagation with pooling, local attention, or long-range skip messages. This transfers the paper's Følner-versus-paradoxical mechanism into an architecture-level…

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Fiberwise amenability of étale groupoids arXiv:2608.09796
△ Mechanism confirmed, baseline not beaten 2026

Tropical Multisymmetric Pooling

Replace ordinary sum or mean pooling in a permutation-invariant set network by the complete family of basic tropical multisymmetric values. For an input set of n points in R^r, each feature computes the maximum total coordinate score obtainable by assigning disjoint rows to prescribed coordinate channels. The resulting representation is invariant to row permutations, separates all multisets, and inherits a bi-Lipschitz relation to optimal row matching, so nearby sets cannot be arbitrarily…

Useful7/10
Difficulty5/10
Novelty8/10
Paper: The basic tropical polynomials generate the semifield of $r$-symmetric tropical rational functions arXiv:2608.06857
✓✓ Beats tuned baseline 2026

Parallel Quadratic Tree Layer

Replace sequential upward message passing in a tree-structured neural module with rake–compress contraction of quadratic latent-state messages. Each node stores a quadratic value function and each edge stores a linear transition or coupling triple; leaf elimination and unary-node compression are implemented as batched Schur complements, followed by a reverse pass that reconstructs node latents and edge outputs. The layer is exact for Gaussian or quadratic latent models and remains…

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Rake-Compress Riccati Recursions for Parallel Scenario-Tree Model Predictive Control arXiv:2608.01332
△ Mechanism confirmed, baseline not beaten 2026

Maximum-Entropy Relational Block Kernel

Parameterize a multi-relational graph kernel as a finite stochastic block model and fit it by maximum entropy subject to differentiable motif-density constraints. Use the resulting block kernel as a graph-neural-network message-passing operator or structured prior for edge prediction, reducing an O(n^2 r) relation tensor to O(m^2 r+n) parameters for m latent blocks and r relations.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Constrained Multi-Relational Graphons with Maximum Entropy arXiv:2607.22383
△ Mechanism confirmed, baseline not beaten 2026

Tangent-Branch Neural Evasion Layer

Wrap a neural multi-agent policy with an analytic planner that generates turn-straight trajectories tangent to pursuer surveillance disks, then selects the branch with the smallest predicted completion time. The network supplies high-level preferences or residual corrections, while the geometric layer prevents unnecessarily entering exclusion regions and exposes an explicit branch-switching signal for training.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Semi-Explicit Solutions to the Prying-Pedestrian Surveillance-Evasion Differential Game and Extensions to Two Pursuers arXiv:2607.21087
△ Mechanism confirmed, baseline not beaten 2026

Invariant-domain learned reconstruction

Insert a neural local reconstruction into a finite-volume or graph-based simulator, but hard-cap its contribution so every reconstructed state remains in the physical admissible set. The network learns accuracy-sensitive gradients or stencil weights; a deterministic limiter, rather than a penalty loss, guarantees positive density and pressure for arbitrary network outputs.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Guarantees by Construction for Learned Finite Volume Schemes on Steady Supersonic Flow arXiv:2607.20171
△ Mechanism confirmed, baseline not beaten 2026

Histogram-Controlled Cluster Updates for Iterative GNNs

Replace node-by-node scheduling in an iterative message-passing network with a learned scheduler that selects one graph cluster at a time, while updating all nodes in that cluster synchronously. The scheduler observes a quantized histogram of local residual weights, making its state invariant to permutations of nodes inside a cluster and independent of cluster cardinality.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Learning to Decode Quantum LDPC Codes via Cluster-Based Sequential Belief Propagation arXiv:2607.20130
✗ Failed on benchmark 2026

Spectral placement of expensive verifiers

Construct a graph of cheap prediction agents or reasoning traces and use a sparse set of expensive verifier calls as graph anchors. Select the next verifier location by the exact reduction in a trace-inverse coherence objective per unit cost, rather than by uncertainty or random sampling. This creates a budgeted mixture-of-agents architecture that can spend computation where it most improves global consensus.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: How Much Does Correctness Cost? Budgeted Placement of Strong Correctors in a Weak Multi-Agent Swarm arXiv:2607.09765
✗ Mechanism failed 2026

Nonlinear Laplacian Equilibrium GNN Layer

Replace several fixed message-passing layers with an implicit graph layer whose state is the solution of a nonlinear flow equilibrium. Learn monotone edge laws from endpoint features, solve for node potentials with damped chord-Newton steps, and use the resulting edge flows or potentials as the layer output. Monotonicity and the Laplacian Jacobian provide a principled stability mechanism while retaining sparse graph computation.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: NLF: A Resistor-Network Framework and Linear-Time Solver for Convex Network-Flow Equilibria arXiv:2607.02041
Unverified 2026

Asynchronous p-Mean Token Mixer

Replace dense token attention with K asynchronous local nonlinear consensus updates on a sparse token graph. Each selected token is moved to the unique incident p-mean of its neighbors, producing a tunable message-passing operator that is averaging for p=2 and nonlinear for other p.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Consensus time for asynchronous $\ell^p$ relaxation: graph dependence arXiv:2609.03856
Unverified 2026

Manifold-Adaptive Positive Quadrature Attention

Replace dense aggregation over patches, points, or graph neighbors with a state-dependent positive quadrature rule. A latent encoder produces the current manifold coordinate z, a small controller chooses a subset of entities, and a convex redistribution step assigns nonnegative weights so that the subset preserves the full aggregation on a learned feature basis. The method is especially suitable for neural operators and vision attention where many spatial entities are redundant along the data…

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Dimensional hyperreduction of nonlinear finite element models via empirical cubature with manifold-adaptive weights arXiv:2609.03068
Unverified 2026

High-girth Bethe assignment layer

Replace independent softmax routing or attention normalization with a differentiable approximate distribution over one-to-one assignments, using the Bethe permanent as the partition-function surrogate. Constrain the allowed token-to-expert or query-to-key support graph to have high girth, which gives an explicit bound on the approximation error and reduces short-cycle-induced correlations.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Optimal girth-dependent bounds for the Bethe approximation of the permanent arXiv:2609.02017
Unverified 2026

Asymmetric Conservative Interaction Layer

Build a neural interaction layer for multiple populations whose cross-population kernels are ordered, so the response of population s to population t need not equal the response of t to s. Evaluate the interaction in divergence form and apply an explicit moment-nullspace projection so each layer preserves total mass, momentum, and energy instead of learning these constraints from penalties.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Learning a general class of admissible multi-species collision operators from molecular dynamics arXiv:2609.01845
Unverified 2026

Mass-Conserving Puncta Router

Insert a differentiable reaction-diffusion layer that converts dense token or pixel features into sparse, spatially coherent routing masks. Two competing orientations form complexes through conserved monomer reservoirs, so local assignments can cluster while opposite assignments mutually exclude one another instead of independently activating at the same location. The layer can be used as a soft-to-hard MoE router, attention-mask generator, or object-part grouping module.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Clustering versus sorting: a mass-conserving reaction-diffusion model of planar polarity puncta arXiv:2608.29679
Unverified 2026

Water-Filled Block-Sparse Neural Connectivity

Partition neural modules into two empirically identified reliability or noise classes and restrict their communication graph to a two-block stochastic block model. Allocate a fixed connectivity budget across within-class and cross-class edges using a water-filling update that favors block pairs producing the largest increase in validation utility. The resulting layer is sparse and modular, with a testable prediction that optimal connectivity concentrates on a few block pairs rather than…

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Graphon Design for Human-Machine Coordination under Bounded Rationality: Optimality of Stochastic Block Models arXiv:2608.27851
Unverified 2026

Leaf-Relocation Tree Router

Learn a branching hierarchy for tokens, examples, or experts by greedily relocating leaves to reduce T-Robinson violations. The resulting tree supplies hierarchical candidate sets for retrieval or MoE routing, allowing the model to search a small subtree instead of all items while adapting the hierarchy to learned representations.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: T-Robinson Spaces: Structure, Recognition, and Applications to Real Data arXiv:2608.21248
Unverified 2026

Budgeted Tree Message-Passing Planner

Before or during graph-neural-network inference, use a tree dynamic program to choose a limited set of active computation nodes and upgraded message-passing edges. A node receives an embedding from an active landmark only when the selected path has enough residual communication radius, so the planner directly optimizes weighted coverage under a joint node-and-edge budget. The resulting active subgraph is then used by a sparse GNN or graph transformer.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Maximum Covering Network Design on Graphs with Low Connectivity: Dynamic Programming and Block-Cut Trees arXiv:2608.20894