Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✓✓ Beats tuned baseline 2026

Adaptive Physics-Lifted Koopman State Space

Replace a purely nonlinear recurrent transition with a learned observable map followed by an explicitly linear latent evolution model. Include the original latent state and a small set of nonlinear observables, and update the linear transition online with forgetting-factor recursive least squares when the environment or task dynamics change.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: Physics-based Online Adaptive Koopman Model Predictive Attitude Control for Combined Spacecraft with Dynamic Uncertainties arXiv:2609.02534
✗ Mechanism failed 2026

Consensus-Corrected Topology-Invariant GNN

Replace ordinary topology-sensitive message passing with scalar-gated aggregation followed by an explicit correction that aligns local node states with a graph-wide consensus component. The correction should make node embeddings less sensitive to line or edge removals while preserving local information needed for prediction. This is suitable for graph neural networks and graph-based world models exposed to changing graph sizes or sparsity patterns.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: UNION: A Unified AC-OPF Framework for Topology-Varying Real-Time Grid Operation arXiv:2608.25784
✗ Failed on benchmark 2026

Non-Abelian Event-Order Memory

Augment an RNN or state-space model with a three-dimensional auxiliary spin updated by noncommuting rotations associated with event types or token classes. The ordered product preserves information that additive counters discard: two sequences with the same number of each event can produce different final spins when their event order differs. Train the spin axes, angles, and readout jointly with the task model while constraining every update to remain on the sphere.

Useful8/10
Difficulty5/10
Novelty8/10
Paper: Non-Abelian Spin Counting of Ordered Stochastic Trajectories: Reentrant Finite-Time Chern Numbers arXiv:2608.23533
✗ Failed on benchmark 2026

Gradient-Flow Commutator Network

Build a neural ODE or invertible transformation whose primitive layers are flows of learned gradient vector fields, then synthesize non-gradient directions using short Lie-bracket commutator products. The paper's bounded-bracket-generation result predicts that restricted gradient primitives can approximate a much larger class of diffeomorphisms than a plain stack of gradient flows.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: The Holonomy of Optimal Mass Transport: The Smooth Case arXiv:2608.15585
△ Mechanism confirmed, baseline not beaten 2026

Differentiable Asymmetric Admissibility Layer

Replace hard clipping or post-hoc asymmetric saturation with a dynamic output state that remains inside a prescribed asymmetric interval. A neural network emits a command uc, while the realized output u evolves through the APIR vector field, producing bounded actions, temporal smoothing, and gradients that remain available in the interior.

Useful8/10
Difficulty4/10
Novelty7/10
Paper: Admissibility-Preserving Control for Strict-Feedback Nonlinear Systems with Asymmetric Actuator Constraints arXiv:2608.15375
✗ Failed on benchmark 2026

Conditioned Irregular-Delay State Encoder

Replace uniformly spaced history taps in a neural state-space encoder with a fixed or learned set of non-uniform delays. Regularize the resulting delay-observation matrix to have a large smallest singular value, which makes latent-state reconstruction less sensitive to irregular timestamps and observation noise. This is directly applicable to event-based data, missing timestamps, and systems with multiple time scales.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Stable Takens' Embedding Theorem for Non-Uniformly-Sampled Linear Systems arXiv:2608.14001
✓✓ Beats tuned baseline 2026

Equivariant Variational Field Network

Represent a scalar energy or free-energy functional of a three-dimensional neural field using translation- and rotation-equivariant convolutions, and produce the field prediction by minimizing the total functional rather than by a direct decoder. The same functional can then generate equilibrium states, forces, and response observables under new external fields, resolutions, and system sizes.

Useful8/10
Difficulty7/10
Novelty6/10
Paper: Equivariant learning of a transferable three-dimensional classical density functional arXiv:2608.13506
✓✓ Beats tuned baseline 2026

Nonlinearity-Subtracted Latent State-Space Model

Build a latent continuous-time neural model with dynamics \(\dot{z}=Az+f_\phi(z)\), where \(f_\phi\) is known, separately computed, or frozen, and \(A\) is learned exclusively from the derivative residual after subtracting \(f_\phi(z)\). Parameterize \(A\) with a truncated SVD or low-rank factorization so its eigenvalues directly predict local stability and long-horizon growth.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Data-driven linear analysis of dynamical systems via nonlinearity-subtracted dynamic mode decomposition arXiv:2608.13373
✓✓ Beats tuned baseline 2026

Inverse-Gain Structured Privileged Distillation

Replace direct action imitation with a causal recurrent estimator of the inverse input gain. The neural network predicts the latent quantity needed by the expert controller, and a fixed algebraic wrapper converts that prediction into an action using the measured state difference and tracking error, thereby removing the additive disturbance exactly under the sampled timing model.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: From Privileged Control to Deployable Adaptation:Fusing Mechanism-Guided Task Reduction with Learned Behavior arXiv:2608.10453
✗ Failed on benchmark 2026

Generator-Flow Equivariance Training

Use discovered infinitesimal generators to create small continuous transformations of hidden states and force a neural predictor to commute with those transformations. This converts symmetry discovery into self-supervised augmentation without prespecifying a group, canonical coordinates, or hand-designed equivariant layers.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: LieStoNet: Learning Lie Symmetries from Spatiotemporal Data for Stochastic Dynamical Systems arXiv:2608.01582
△ Mechanism confirmed, baseline not beaten 2026

Stable Modal State-Space Bottleneck

Insert a continuous spatial trunk and a pole-constrained modal state-space branch into a spatiotemporal predictor. The model represents a field as a sum of learned spatial modes and exponentially evolving modal coordinates, so long-horizon behavior is controlled by explicit poles rather than by an unconstrained recurrent transition matrix.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Learning Structural Eigenmodes with Modal Operator Network (ModalONet) arXiv:2607.28926
△ Mechanism confirmed, baseline not beaten 2026

Belief-Intersection Forcing Shield

Add a discrete-event safety shield between a partially observed neural policy and the environment. The policy proposes a forcing action, but the shield permits it only when the same decision is safe for every latent plant state compatible with the current observation; otherwise it returns a certified inconsistency or a conservative fallback. This converts forcing consistency into an implementable robust action-selection rule rather than trusting a single estimated hidden state.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Supervisory Control with Event Forcing Under Partial Observation arXiv:2607.21040
△ Mechanism confirmed, baseline not beaten 2026

Edge-Supported Polynomial State Space

Replace a complete tensor/Kronecker polynomial lift of a graph dynamical system with observables selected only from the support of the interaction graph. The lifted state can then be propagated by a sparse structured linear operator, while the first omitted degree is treated as an explicit residual or learned closure. This gives a graph-aware polynomial state-space layer for neural ODEs, graph RNNs, and world models.

Useful8/10
Difficulty5/10
Novelty8/10
Paper: Graph-Induced Tensor Liftings for Networked SEIR Models: Dimensional Reduction and Residual Analysis arXiv:2607.17664
✗ Failed on benchmark 2026

Contractive Latent Observer

Replace recurrence or nearest-neighbour analogue lookup with a learned delay-coordinate observer that continuously corrects a latent state using the current observation. Constrain the observer's closed-loop Jacobian or linear state matrix to have spectral radius below one, so prediction error contracts geometrically and required burn-in grows logarithmically with target accuracy.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: Contraction versus Recurrence: An Exponential Separation in Observation-Based Prediction of Deterministic Dynamics arXiv:2607.14885
△ Mechanism confirmed, baseline not beaten 2026

Nilpotent BGG Neural Complex

Replace an unconstrained stack of learned vector-field or tensor-field maps by a short neural complex whose fixed differential operators satisfy D_{k+1}D_k=0. The network predicts potentials or quotient representatives, making curl-of-gradient, divergence-of-curl, compatibility, and gauge constraints exact rather than penalty-based.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: The Bernstein-Gelfand-Gelfand (BGG) Construction: Algebra, Geometry, and Analysis; Part I arXiv:2607.10662
△ Mechanism confirmed, baseline not beaten 2026

Accumulator-Carrying Picard ResNet

Build a residual module whose state explicitly contains both a persistent context representation and an accumulator. Each residual branch computes one learned correction and adds it to the accumulator, instead of forcing every layer to represent the complete output from scratch. This provides a concrete solver-like architecture for high-dimensional regression and iterative latent prediction.

Useful7/10
Difficulty4/10
Novelty5/10
Paper: Residual neural networks overcome the curse of dimensionality for semilinear heat equations arXiv:2609.03626
✗ Failed on benchmark 2026

Rankine–Hugoniot Front Tokens

Augment a 1D neural operator or transformer with explicit tokens for detected discontinuities. Advance each front analytically using the local Rankine–Hugoniot speed and train the network only to reconstruct smooth regions and the residual caused by source terms and grid resolution.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Physics-informed token transformer methodology for nonlinear balance laws. I. Schwarzschild--Burgers fluid flows arXiv:2607.23143
△ Mechanism confirmed, baseline not beaten 2026

Nullspace-coordinate constrained operator blocks

Build a neural operator from frozen ambient mechanism blocks and a geometry-specific algebraic constraint adapter. The adapter parameterizes all outputs in the affine set satisfying sampled linear constraints exactly, so the network never produces boundary-violating states and does not require a penalty coefficient or post-step projection.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Geometry-aware LegONet for PDE Learning on Arbitrary Domains arXiv:2607.23069
△ Mechanism confirmed, baseline not beaten 2026

Conservative Parallel-Edge Decomposition

Represent a multi-input interaction by several single-input edge channels and enforce conservation only after their signed contributions are summed at the vertices. This provides a neural architecture for composite interactions in which different channels have different drivers, while preventing the node update from inventing or destroying net internal flow.

Useful7/10
Difficulty4/10
Novelty5/10
Paper: Multi-Domain Graph-Based Modeling of Energy Systems with Applications to Lithium-Ion Batteries arXiv:2608.30157
✗ Failed on benchmark 2026

Fractional Memory State-Space Layer

Replace a standard recurrent state update or finite-order SSM filter with a causal relative-history operator using a weakly singular kernel k(s)=s^{p-1}m(s), where 0<p<1. The resulting layer retains information over a power-law range of timescales and introduces tunable frequency-dependent phase and attenuation, while remaining implementable through a small bank of exponentially decaying states.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: High frequency wave propagation for the viscoelastic wave equation with singular memory arXiv:2608.30138
△ Mechanism confirmed, baseline not beaten 2026

Hypoelliptic transport-diffusion layer

Replace an isotropic local mixing layer with a kinetic layer that smooths features in x and transports them in y along the characteristic direction x. The layer should be useful for phase-space data, learned simulators, and world models in which positions or transported quantities evolve through coupled drift and diffusion rather than independent Euclidean motion.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Boundary Harnack inequalities for Kolmogorov equations in asymptotically cylindrical Lipschitz domains arXiv:2608.29813
△ Mechanism confirmed, baseline not beaten 2026

Monotone Compositional Reachability Critic

Train separate neural value functions for primitive reachability, avoidance, or target-reaching tasks, then combine them with a coordinatewise monotone aggregator whose derivatives with respect to all primitive values are nonnegative. This transfers the paper's exact two-player decomposition condition into a modular critic architecture: adding a new target changes only one primitive critic and the aggregator, rather than requiring a new high-dimensional value function.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Exact Decomposition of Value Functions for Two-Player Games in Hamilton-Jacobi Reachability arXiv:2608.27654
✗ Failed on benchmark 2026

Equation-addressable equilibrium layer

Build a recurrent or implicit neural layer from a bipartite graph containing variable nodes and equation or mechanism nodes, rather than a directed graph containing only variables. The forward pass solves all mechanism residuals simultaneously, while an intervention replaces one selected equation and fixes its target variable; this distinguishes interventions that impose the same value through different mechanisms. The resulting module is suitable for equilibrium world models, differentiable…

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Causal Reasoning with Bipartite Graphical Causal Models arXiv:2608.19831
✗ Failed on benchmark 2026

Hyperplane-Gated Piecewise Neural Dynamics

Replace a single smooth neural vector field with a finite collection of smooth subnetworks selected by learned affine hyperplanes. The architecture exposes switching geometry directly, allowing it to represent friction-like or threshold dynamics without approximating discontinuities using excessively steep activations.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Learning piecewise-smooth dynamical systems arXiv:2608.19785