Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✗ Failed on benchmark 2026

Rankine–Hugoniot Front Tokens

Augment a 1D neural operator or transformer with explicit tokens for detected discontinuities. Advance each front analytically using the local Rankine–Hugoniot speed and train the network only to reconstruct smooth regions and the residual caused by source terms and grid resolution.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Physics-informed token transformer methodology for nonlinear balance laws. I. Schwarzschild--Burgers fluid flows arXiv:2607.23143
△ Mechanism confirmed, baseline not beaten 2026

Nullspace-coordinate constrained operator blocks

Build a neural operator from frozen ambient mechanism blocks and a geometry-specific algebraic constraint adapter. The adapter parameterizes all outputs in the affine set satisfying sampled linear constraints exactly, so the network never produces boundary-violating states and does not require a penalty coefficient or post-step projection.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Geometry-aware LegONet for PDE Learning on Arbitrary Domains arXiv:2607.23069
✓✓ Beats tuned baseline 2026

Spectral pinning of neural modules

Represent communicating layers, experts, or distributed workers as nodes of a weighted graph and apply strong corrective updates only to a small pinned subset. Select pins by the increase they produce in the grounded Laplacian smallest eigenvalue, because this spectral gap predicts the decay rate of representation disagreement.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Event-Triggered Pinning Impulsive Control of Complex Networks with Actuation Delays: Stability Analysis and Zeno-Free Conditions arXiv:2608.24074
✗ Failed on benchmark 2026

Holonomy-designed recurrent memory

Build a recurrent module from two finite-state factors: a normalization state and a winner-selection state. Choose or learn their coupling so that the joint transition system contains a certified composite two-cycle, giving the network a small robust memory state, while every fixed-input generator still collapses most states toward attractors. The module can be embedded in a continuous RNN using soft state assignments during training and straight-through discretization for algebraic auditing.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: "More Is Different'' in Neural Circuits: Algebraic Emergence of Effective Theories in Canonical Recurrent Motifs of Biological Neuronal Networks arXiv:2608.30231
△ Mechanism confirmed, baseline not beaten 2026

Conservative Parallel-Edge Decomposition

Represent a multi-input interaction by several single-input edge channels and enforce conservation only after their signed contributions are summed at the vertices. This provides a neural architecture for composite interactions in which different channels have different drivers, while preventing the node update from inventing or destroying net internal flow.

Useful7/10
Difficulty4/10
Novelty5/10
Paper: Multi-Domain Graph-Based Modeling of Energy Systems with Applications to Lithium-Ion Batteries arXiv:2608.30157
✗ Failed on benchmark 2026

Fractional Memory State-Space Layer

Replace a standard recurrent state update or finite-order SSM filter with a causal relative-history operator using a weakly singular kernel k(s)=s^{p-1}m(s), where 0<p<1. The resulting layer retains information over a power-law range of timescales and introduces tunable frequency-dependent phase and attenuation, while remaining implementable through a small bank of exponentially decaying states.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: High frequency wave propagation for the viscoelastic wave equation with singular memory arXiv:2608.30138
✗ Mechanism failed 2026

Orbit-Consistent Equivariant Distillation

Constrain a student policy to transform its action in the same way that the input state is transformed, while constraining its value estimate to remain unchanged. During distillation, augment every teacher-student pair with several symmetry-transformed copies and penalize disagreement after transforming the student action back to the original frame.

Useful7/10
Difficulty5/10
Novelty4/10
Paper: SymVD: Symmetric Vision Language Action Distillation for Robot Manipulation arXiv:2608.29828
△ Mechanism confirmed, baseline not beaten 2026

Hypoelliptic transport-diffusion layer

Replace an isotropic local mixing layer with a kinetic layer that smooths features in x and transports them in y along the characteristic direction x. The layer should be useful for phase-space data, learned simulators, and world models in which positions or transported quantities evolve through coupled drift and diffusion rather than independent Euclidean motion.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Boundary Harnack inequalities for Kolmogorov equations in asymptotically cylindrical Lipschitz domains arXiv:2608.29813
△ Mechanism confirmed, baseline not beaten 2026

Positive-Regime Observable ReLU State Space

Constrain recurrent preactivations to remain nonnegative so that ReLU acts as the identity along realized trajectories. The hidden dynamics then admit a classical linear observability matrix, allowing principled hidden-coordinate selection and conditioning control instead of relying on potentially destructive activation masks.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: On the Number of Observation Nodes in Recurrent Neural Networks with Linear Threshold and ReLU Functions arXiv:2608.29650
✓✓ Beats tuned baseline 2026

Fourier-Mode Stability Shaping

Use a cyclic forward-neighbor recurrent or state-space layer and regularize its coupling so selected discrete Fourier modes are contracting while task-critical modes remain weakly damped. The paper's exact mode factors make instability falsifiable: a mode becomes unstable when its scalar factor changes sign, producing a measurable transition rather than a vague smoothness prior.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: Existence and Stability of Dancing Equilibria in Asymmetric Kuramoto Networks arXiv:2608.29630
△ Mechanism confirmed, baseline not beaten 2026

Task-Tangent Capture Pruning

Prune parameter directions according to how much task-relevant Jacobian energy they carry, rather than by weight magnitude or individual gradient magnitude. Keep a mask whose discarded tangent component is at most an empirical fraction epsilon of the full tangent vector for calibration task directions, thereby preserving the local output dynamics seen by the task.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: The role of parameter Jacobians in the stability of network outputs arXiv:2608.27748
△ Mechanism confirmed, baseline not beaten 2026

Monotone Compositional Reachability Critic

Train separate neural value functions for primitive reachability, avoidance, or target-reaching tasks, then combine them with a coordinatewise monotone aggregator whose derivatives with respect to all primitive values are nonnegative. This transfers the paper's exact two-player decomposition condition into a modular critic architecture: adding a new target changes only one primitive critic and the aggregator, rather than requiring a new high-dimensional value function.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Exact Decomposition of Value Functions for Two-Player Games in Hamilton-Jacobi Reachability arXiv:2608.27654
✗ Failed on benchmark 2026

Topological Fluctuation Graph Layer

Replace a deterministic graph propagation layer by a stable stochastic linearized latent dynamics whose frequency-resolved covariance matrix defines spectral bands. Train or initialize the graph operator so that a selected covariance band has a nonzero Chern number and remains separated by a measurable spectral gap, producing representations that are robust to local perturbations and can support boundary-localized responses.

Useful7/10
Difficulty7/10
Novelty8/10
Paper: Topology of Fluctuation Bands in Chiral Active Matter arXiv:2608.26055
△ Mechanism confirmed, baseline not beaten 2026

Critical Cross-Layer Weight Sharing

Construct deep or recurrent networks whose layer weights are correlated across depth with a prescribed power-law covariance, rather than either fully tying or fully independently sampling layers. The paper predicts two usable design boundaries: \(\gamma=1/2\) for divergence of correlation-induced fourth moments and \(\gamma=1\) for loss of summable-correlation flatness.

Useful7/10
Difficulty6/10
Novelty8/10
Paper: Bulk Phase Transition and Edge Behavior in Temporally Correlated Random Matrices arXiv:2608.23944
✗ Mechanism failed 2026

Fractional-memory recurrent state

Construct an efficient recurrent or state-space layer whose impulse response follows Mittag-Leffler relaxation instead of a single exponential. A bank of stable diagonal state channels approximates the long power-law tail, allowing the layer to retain information over widely separated timescales with only \(K\) states per feature.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Anomalous diffusion memory factorization: Characteristic timescales and application to inverse problem arXiv:2608.21674
✓✓ Beats tuned baseline 2026

PPMI-Gated BCM Sparse Graph Encoder

Replace a dense graph embedding table or end-to-end GNN encoder with a fixed-width binary SDR learned from streaming random-walk context pairs. Use PPMI to amplify informative node-context pairs and a local BCM update to learn detector columns, followed by k-winner-take-all binarization. The resulting sparse code can be used directly for node classification, link prediction, retrieval, or as input to a small downstream predictor.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Columnar-Embedder: A Biologically Inspired Cortical Architecture for Binary Sparse Distributed Graph Representations arXiv:2608.20408
✗ Failed on benchmark 2026

Equation-addressable equilibrium layer

Build a recurrent or implicit neural layer from a bipartite graph containing variable nodes and equation or mechanism nodes, rather than a directed graph containing only variables. The forward pass solves all mechanism residuals simultaneously, while an intervention replaces one selected equation and fixes its target variable; this distinguishes interventions that impose the same value through different mechanisms. The resulting module is suitable for equilibrium world models, differentiable…

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Causal Reasoning with Bipartite Graphical Causal Models arXiv:2608.19831
✗ Failed on benchmark 2026

Hyperplane-Gated Piecewise Neural Dynamics

Replace a single smooth neural vector field with a finite collection of smooth subnetworks selected by learned affine hyperplanes. The architecture exposes switching geometry directly, allowing it to represent friction-like or threshold dynamics without approximating discontinuities using excessively steep activations.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Learning piecewise-smooth dynamical systems arXiv:2608.19785
△ Mechanism confirmed, baseline not beaten 2026

Quadrature-Whitened Neural Feature Subspace

Freeze a wide neural spatial dictionary, then compress and whiten it using the quadrature mass matrix before solving for output coefficients or latent PDE states. The retained basis removes feature directions that are numerically invisible or nearly dependent under the actual domain discretization, while preserving the represented function space up to the chosen SVD rank.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Evo-GTransNet for Parabolic PDEs: A Fixed-Feature Galerkin Method of Lines with Quadrature-Mass Orthonormalization arXiv:2608.19615
✗ Failed on benchmark 2026

Quadratic center-manifold bottleneck

Replace a conventional linear decoder in an autoencoder or latent state-space model with an explicit quadratic manifold decoder, allowing a small latent vector to represent curved and transport-like state trajectories. Add a dynamics-aware invariance loss that penalizes the discrepancy between the time derivative of the quadratic manifold and the neural dynamics evaluated on that manifold.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Beyond linear subspaces: Nonlinear moment matching meets quadratic manifolds arXiv:2608.19486
△ Mechanism confirmed, baseline not beaten 2026

Harmonic-Mode Branch for Topological Memory

Do not force Hodge dissipation onto harmonic edge modes, because these modes are precisely the obstruction to global coercivity. Split the latent state into dissipative coexact modes and a finite-dimensional harmonic branch, and use harmonic-decoupled interactions so each harmonic coordinate defines an invariant affine fibre with its own attractor.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Hodge Coercivity and Global Dynamics in Two-Field Edge-Cochain Systems with MHD-Type Cancellation arXiv:2608.19360
✗ Mechanism failed 2026

Automaton-Supervised Neural Policy Shield

Attach a deterministic supervisory automaton to a neural policy or sequence model and mask every event disabled by the current supervisor state. Use a short receding-horizon planner over admissible events to resolve conflicts between neural preferences and shared-resource constraints. The network scores useful actions, while the automaton supplies an exact safety layer.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Model Predictive Supervisory Control for Hierarchical and Distributed UAS Traffic Management arXiv:2608.18353
△ Mechanism confirmed, baseline not beaten 2026

Ordered Diffusion Message Passing

Use a learned scalar ordering function to turn a symmetric local Gaussian graph kernel into a directed, row-stochastic message-passing operator. The asymmetric tilt lets neighboring nodes communicate preferentially along an inferred dynamical direction, while the Gaussian factor retains locality and diffusion-like smoothing.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Ordered Diffusion Kernels arXiv:2608.18019
△ Mechanism confirmed, baseline not beaten 2026

Koopman Hankel Dual Autoencoder

Replace pointwise sequence reconstruction with reconstruction of overlapping past and future Hankel windows in a shared latent manifold. A first encoder compresses the delay-coordinate trajectory, while a second decoder or predictor reconstructs the future block from the latent state; training therefore penalizes representations that fit observations but do not preserve dynamical evolution.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Fault detection on manifolds of nonlinear dynamical systems with dual autoencoders arXiv:2608.17698