Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✗ Failed on benchmark 2026

Small-gain certified modular network

Partition a neural network into independently trained or independently monitored modules and constrain their cross-module interaction gain using a compositional contraction certificate. This enables stable deep modular MLPs, graph blocks, or recurrent modules without estimating the full network Jacobian, while providing an explicit coupling threshold for when the architecture loses contraction.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Contraction Certification from Streaming Data: Wasserstein Robustness and Compositional Stability for Interconnected Nonlinear System arXiv:2607.11982
✗ Failed on benchmark 2026

Topology-Aware Streaming Jacobian Monitor

For a recurrent or graph neural network with known local connectivity, estimate each node's local Jacobian row using only graph neighbors rather than all hidden coordinates. Use the resulting sparse Jacobian both to compute a contraction certificate and to regularize training toward dynamically local interactions, reducing estimator variance and the number of samples required for reliable stability decisions.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Streaming Contraction Certificates for Nonlinear Networks: Topology-Aware Data Sufficiency with Partial Observation arXiv:2607.10893
✗ Failed on benchmark 2026

Topological Fluctuation Graph Layer

Replace a deterministic graph propagation layer by a stable stochastic linearized latent dynamics whose frequency-resolved covariance matrix defines spectral bands. Train or initialize the graph operator so that a selected covariance band has a nonzero Chern number and remains separated by a measurable spectral gap, producing representations that are robust to local perturbations and can support boundary-localized responses.

Useful7/10
Difficulty7/10
Novelty8/10
Paper: Topology of Fluctuation Bands in Chiral Active Matter arXiv:2608.26055
✗ Failed on benchmark 2026

Shared-response expert ranking

Build a label-free router for a finite library of neural operators by estimating one shared physical target response from an anchor prediction and using it to rank every candidate through inner products with candidate differences. The method avoids running a full residual-based diagnostic independently for every expert and can be used either to select the best expert or to form a corrected weighted combination.

Useful7/10
Difficulty6/10
Novelty8/10
Paper: Shared Physics Responses Recover Hidden Rankings in Neural Operator Libraries arXiv:2608.20441
✗ Failed on benchmark 2026

Equation-addressable equilibrium layer

Build a recurrent or implicit neural layer from a bipartite graph containing variable nodes and equation or mechanism nodes, rather than a directed graph containing only variables. The forward pass solves all mechanism residuals simultaneously, while an intervention replaces one selected equation and fixes its target variable; this distinguishes interventions that impose the same value through different mechanisms. The resulting module is suitable for equilibrium world models, differentiable…

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Causal Reasoning with Bipartite Graphical Causal Models arXiv:2608.19831
✗ Failed on benchmark 2026

Gauge-Covariant Wilson-Loop Regularization

Attach an SU(2) transport matrix to every directed edge of a graph neural network and penalize nontrivial plaquette holonomies instead of penalizing individual edge transformations. The regularizer is invariant to arbitrary local changes of latent representation frame, encouraging path-consistent relational features without requiring all edges to share one global coordinate system.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Frustration without Glass: A Non-Abelian Gauge Model of Network Compatibility arXiv:2608.17817
✗ Failed on benchmark 2026

Gauge-Free Spectral OT Layer

Parameterize an entropic OT cost only in directions that can change the transport plan, removing row-plus-column potential directions that are invisible because of OT gauge invariance. Whiten the remaining feature coordinates using their empirical covariance, producing an OT layer whose identifiable parameters have substantially more uniform sensitivity.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Sinkhorn Linearization and the Spectral Proxy: Unifying the Statistical and Algorithmic Theory of Feature-Parameterized Inverse Optimal Transport via a Single Spectral Sandwich arXiv:2608.13201
✗ Failed on benchmark 2026

Følner-Gated Message Passing

Add an online receptive-field expansion monitor to a graph neural network and use it to gate message-passing depth or invoke graph pooling. For a sampled node set F and propagation neighborhood K, continue fine-scale propagation only while the growth ratio |KF|/|F| is close to one; when it is persistently expansive, replace further propagation with pooling, local attention, or long-range skip messages. This transfers the paper's Følner-versus-paradoxical mechanism into an architecture-level…

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Fiberwise amenability of étale groupoids arXiv:2608.09796
✗ Failed on benchmark 2026

Differentiable Persistence Landscape Layer

Convert each persistence diagram produced from an input, intermediate feature map, or graph filtration into a discretized persistence landscape and feed it to an MLP or concatenate it with ordinary neural features. Unlike a variable-size list of birth-death pairs, the landscape has a fixed tensor shape and is provably nonexpansive with respect to the diagram Wasserstein distance.

Useful7/10
Difficulty4/10
Novelty5/10
Paper: A Hilbert space embedding of persistence diagrams and barcodes arXiv:2608.08858
✗ Failed on benchmark 2026

Volume-Mass Diffusion GNN

Replace ordinary graph propagation by diffusion with a positive node-dependent mass matrix \(\mathbf V\), so high-volume nodes update slowly and low-volume nodes update rapidly. Use node volumes as fixed metadata, a function of degree, or learned positive gates; this makes the architecture sensitive to dynamical localization that degree-normalized GCNs cannot represent.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Localization transitions of diffusion dynamics in physical networks arXiv:2607.19486
✗ Failed on benchmark 2026

ZCA In-Context Output Transport

Train a neural surrogate to predict outputs in a source-domain ZCA-whitened space, then adapt to a shifted domain using only the shifted domain's output mean and covariance. At inference, transport the network prediction through the target covariance square root, yielding a weight-free correction that preserves output-coordinate semantics and can be applied to MLP, CNN, graph-NN, or transformer regressors.

Useful7/10
Difficulty4/10
Novelty5/10
Paper: Gradient-Free Topology Adaptation for Power Flow Surrogates via In-Context Whitening arXiv:2607.12241
✗ Failed on benchmark 2026

Causal-Masked Neural SDE

Replace a fully connected neural SDE drift with coordinate-wise functions that can read only the paths of graph parents. Learn soft edge gates and penalize violations of the paper's pathwise Lipschitz condition, so the model remains stable during long rollouts and supports explicit interventions on selected coordinates.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Causal Graphs, Markov Properties and Do-calculus for Stochastic Differential Equations arXiv:2607.12140
✗ Failed on benchmark 2026

Spectral placement of expensive verifiers

Construct a graph of cheap prediction agents or reasoning traces and use a sparse set of expensive verifier calls as graph anchors. Select the next verifier location by the exact reduction in a trace-inverse coherence objective per unit cost, rather than by uncertainty or random sampling. This creates a budgeted mixture-of-agents architecture that can spend computation where it most improves global consensus.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: How Much Does Correctness Cost? Budgeted Placement of Strong Correctors in a Weak Multi-Agent Swarm arXiv:2607.09765
✗ Failed on benchmark 2026

Equilibrium-Coordinate Neural Operator

Build a neural PDE surrogate that predicts changes in equilibrium variables rather than changes in conservative state variables. The network receives the local state and geometry, predicts an equilibrium-coordinate increment, and subtracts the network output evaluated at a reference equilibrium, forcing the reference state to have exactly zero learned residual.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Fifth-Order Well-Balanced Path-Conservative A-WENO Scheme for the Ripa Model arXiv:2607.09293
✗ Failed on benchmark 2026

Conservative Sparse Mortar Cross-Attention

Replace dense coarse-to-fine cross-attention at multiresolution interfaces with a sparse, nonnegative overlap operator whose weighted feature average is exactly conserved between the two resolutions. Use this operator as a low-order path and blend it with an unrestricted neural cross-attention path through a convex limiter that keeps features inside a box or simplex domain. The construction is especially suitable for adaptive token grids, hierarchical graph neural networks, neural operators…

Useful7/10
Difficulty5/10
Novelty8/10
Paper: Invariant-domain-preserving limiting with Adaptive Mesh Refinement for Legendre-Gauss-Lobatto Discontinuous Galerkin Spectral Element Methods arXiv:2607.06045
✗ Failed on benchmark 2026

Two-Scale Parabolic Filter Block

Construct a spatiotemporal neural block from localized functions of a learned parabolic operator instead of unrestricted attention or convolution. Use one filter for fine-scale diffusion and another for coarse-scale temporal aggregation, with the scale ratio controlling information propagation. The block should suppress distant interactions while still permitting long-range mixing through coarse filters.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: $\mathrm{L}^p$ bounds for parabolic Riesz transforms with rough coefficients: The case $1<p \leq 2$ arXiv:2607.05181
✗ Failed on benchmark 2026

Differentiable Gaussian DAG Layer

Replace an unconstrained covariance or dependency module with a topologically ordered linear-Gaussian DAG whose edge transforms and innovation covariances are neural-network parameters. The layer computes a joint covariance by a differentiable triangular solve, allowing downstream losses to use uncertainty, conditional prediction, or dependency penalties while preserving positive semidefiniteness by construction. This is especially suitable for graph neural networks, structured VAEs, and…

Useful7/10
Difficulty5/10
Novelty6/10
Paper: A Differentiable Covariance Calculus for Linear Gaussian Bayesian Networks arXiv:2607.04578
✗ Failed on benchmark 2026

Cross-domain amortized inverse operator

Use separate measurement-domain and target-domain token sets so a network can infer a field on one spatial domain from sparse observations on another in one forward pass. The same decoder can answer arbitrary target query points, avoiding an optimization loop for each inverse instance.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: GAIA: Geometry-Adaptive Operator Learning for Forward and Inverse Problems arXiv:2607.01128
✗ Failed on benchmark 2026

Inverse-Square Fractional Attention

Replace or augment geometric attention on spatial or point-cloud tokens with a positive fractional kernel containing the paper's inverse-square origin factor. This gives tokens near a designated singular center a controlled increase in receptive-field influence while preserving a scale-invariant distance decay, which may help models represent cusp-like fields, radial singularities, and multiscale spatial interactions.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Sharp and Endpoint Two-Weight Fractional Integral Estimates for Schr"odinger Operators with Inverse-Square Potentials arXiv:2607.09585
✗ Failed on benchmark 2026

Wide-tree invariant alignment layer

Replace a purely pairwise embedding similarity used for set alignment with a sum of rooted-tree contraction scores. Each tree feature aggregates products of several coordinate-level interactions and can preserve correspondence information under an unknown orthogonal transformation, allowing matching from moderate correlation rather than nearly identical embeddings. Use the resulting score matrix for Hungarian matching, contrastive loss, or a differentiable Sinkhorn assignment.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: High-Dimensional Procrustes Matching via Tree Counts arXiv:2607.08538
✗ Failed on benchmark 2026

Lipschitz-Free Metric Pooling

Replace coordinate-wise mean pooling of metric-valued items with a finite representation of their free integral. Each item x in a pointed metric space M is represented through evaluations of learned Lipschitz probes, and the pooled feature is the weighted integral of those probe values. A dual Lipschitz critic estimates the free-space norm of differences between pooled groups, making the representation sensitive to metric geometry while remaining permutation-invariant.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Analytic integration of metric-valued functions in Lipschitz free spaces arXiv:2607.06049
✗ Failed on benchmark 2026

Orthogonal-Rank Contextual Memory

Replace a discrete or one-hot recurrent state table with a low-dimensional vector memory whose event embeddings are orthogonal whenever the corresponding events are mutually exclusive in an input exclusivity graph. The module uses continuous state vectors and can therefore target dimension \(d=\xi(G)\), whereas a discrete state encoding is lower-bounded by \(N\geq\chi(G)\). This should be tested on graph-defined formal-language recognition tasks, where the graph is known and the claimed…

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Quantum Memory Advantage from Contextuality arXiv:2607.00507