Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

△ Mechanism confirmed, baseline not beaten 2026

Equivariant Shared-Mechanism World Model

Use the paper's families of local graph embeddings to identify repeated occurrences of the same causal substructure across time steps, environments, or entities. Feed every aligned occurrence through one shared transition mechanism and impose an explicit equivariance penalty under the symmetry group acting on occurrence indices, rather than learning an independent predictor for every context.

Useful8/10
Difficulty5/10
Novelty5/10
Paper: Symmetries and Causality: Causal Effect Identification Beyond IID Data arXiv:2609.03697
✓✓ Beats tuned baseline 2026

Adaptive Physics-Lifted Koopman State Space

Replace a purely nonlinear recurrent transition with a learned observable map followed by an explicitly linear latent evolution model. Include the original latent state and a small set of nonlinear observables, and update the linear transition online with forgetting-factor recursive least squares when the environment or task dynamics change.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: Physics-based Online Adaptive Koopman Model Predictive Attitude Control for Combined Spacecraft with Dynamic Uncertainties arXiv:2609.02534
✗ Mechanism failed 2026

Consensus-Corrected Topology-Invariant GNN

Replace ordinary topology-sensitive message passing with scalar-gated aggregation followed by an explicit correction that aligns local node states with a graph-wide consensus component. The correction should make node embeddings less sensitive to line or edge removals while preserving local information needed for prediction. This is suitable for graph neural networks and graph-based world models exposed to changing graph sizes or sparsity patterns.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: UNION: A Unified AC-OPF Framework for Topology-Varying Real-Time Grid Operation arXiv:2608.25784
✗ Failed on benchmark 2026

Non-Abelian Event-Order Memory

Augment an RNN or state-space model with a three-dimensional auxiliary spin updated by noncommuting rotations associated with event types or token classes. The ordered product preserves information that additive counters discard: two sequences with the same number of each event can produce different final spins when their event order differs. Train the spin axes, angles, and readout jointly with the task model while constraining every update to remain on the sphere.

Useful8/10
Difficulty5/10
Novelty8/10
Paper: Non-Abelian Spin Counting of Ordered Stochastic Trajectories: Reentrant Finite-Time Chern Numbers arXiv:2608.23533
✗ Failed on benchmark 2026

Gradient-Flow Commutator Network

Build a neural ODE or invertible transformation whose primitive layers are flows of learned gradient vector fields, then synthesize non-gradient directions using short Lie-bracket commutator products. The paper's bounded-bracket-generation result predicts that restricted gradient primitives can approximate a much larger class of diffeomorphisms than a plain stack of gradient flows.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: The Holonomy of Optimal Mass Transport: The Smooth Case arXiv:2608.15585
△ Mechanism confirmed, baseline not beaten 2026

Differentiable Asymmetric Admissibility Layer

Replace hard clipping or post-hoc asymmetric saturation with a dynamic output state that remains inside a prescribed asymmetric interval. A neural network emits a command uc, while the realized output u evolves through the APIR vector field, producing bounded actions, temporal smoothing, and gradients that remain available in the interior.

Useful8/10
Difficulty4/10
Novelty7/10
Paper: Admissibility-Preserving Control for Strict-Feedback Nonlinear Systems with Asymmetric Actuator Constraints arXiv:2608.15375
✗ Failed on benchmark 2026

Conditioned Irregular-Delay State Encoder

Replace uniformly spaced history taps in a neural state-space encoder with a fixed or learned set of non-uniform delays. Regularize the resulting delay-observation matrix to have a large smallest singular value, which makes latent-state reconstruction less sensitive to irregular timestamps and observation noise. This is directly applicable to event-based data, missing timestamps, and systems with multiple time scales.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Stable Takens' Embedding Theorem for Non-Uniformly-Sampled Linear Systems arXiv:2608.14001
✓✓ Beats tuned baseline 2026

Equivariant Variational Field Network

Represent a scalar energy or free-energy functional of a three-dimensional neural field using translation- and rotation-equivariant convolutions, and produce the field prediction by minimizing the total functional rather than by a direct decoder. The same functional can then generate equilibrium states, forces, and response observables under new external fields, resolutions, and system sizes.

Useful8/10
Difficulty7/10
Novelty6/10
Paper: Equivariant learning of a transferable three-dimensional classical density functional arXiv:2608.13506
✓✓ Beats tuned baseline 2026

Nonlinearity-Subtracted Latent State-Space Model

Build a latent continuous-time neural model with dynamics \(\dot{z}=Az+f_\phi(z)\), where \(f_\phi\) is known, separately computed, or frozen, and \(A\) is learned exclusively from the derivative residual after subtracting \(f_\phi(z)\). Parameterize \(A\) with a truncated SVD or low-rank factorization so its eigenvalues directly predict local stability and long-horizon growth.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Data-driven linear analysis of dynamical systems via nonlinearity-subtracted dynamic mode decomposition arXiv:2608.13373
✓✓ Beats tuned baseline 2026

Inverse-Gain Structured Privileged Distillation

Replace direct action imitation with a causal recurrent estimator of the inverse input gain. The neural network predicts the latent quantity needed by the expert controller, and a fixed algebraic wrapper converts that prediction into an action using the measured state difference and tracking error, thereby removing the additive disturbance exactly under the sampled timing model.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: From Privileged Control to Deployable Adaptation:Fusing Mechanism-Guided Task Reduction with Learned Behavior arXiv:2608.10453
△ Mechanism confirmed, baseline not beaten 2026

Stable Modal State-Space Bottleneck

Insert a continuous spatial trunk and a pole-constrained modal state-space branch into a spatiotemporal predictor. The model represents a field as a sum of learned spatial modes and exponentially evolving modal coordinates, so long-horizon behavior is controlled by explicit poles rather than by an unconstrained recurrent transition matrix.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Learning Structural Eigenmodes with Modal Operator Network (ModalONet) arXiv:2607.28926
△ Mechanism confirmed, baseline not beaten 2026

Belief-Intersection Forcing Shield

Add a discrete-event safety shield between a partially observed neural policy and the environment. The policy proposes a forcing action, but the shield permits it only when the same decision is safe for every latent plant state compatible with the current observation; otherwise it returns a certified inconsistency or a conservative fallback. This converts forcing consistency into an implementable robust action-selection rule rather than trusting a single estimated hidden state.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Supervisory Control with Event Forcing Under Partial Observation arXiv:2607.21040
✗ Mechanism failed 2026

Matrix-Free Differentiable CBF Safety Layer

Attach a hard control-barrier-function quadratic-program safety filter to a neural policy, but solve the filter with operator splitting and differentiate through its fixed-point map using projection Jacobian-vector products. The network learns the nominal action and task objective end to end, while the deployed action remains the feasible filtered action rather than an unconstrained penalty-based approximation.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: End-to-End Learning of Safe Optimal Feedback Control in High Dimensions with Control Barrier Function Layers arXiv:2607.20674
△ Mechanism confirmed, baseline not beaten 2026

Edge-Supported Polynomial State Space

Replace a complete tensor/Kronecker polynomial lift of a graph dynamical system with observables selected only from the support of the interaction graph. The lifted state can then be propagated by a sparse structured linear operator, while the first omitted degree is treated as an explicit residual or learned closure. This gives a graph-aware polynomial state-space layer for neural ODEs, graph RNNs, and world models.

Useful8/10
Difficulty5/10
Novelty8/10
Paper: Graph-Induced Tensor Liftings for Networked SEIR Models: Dimensional Reduction and Residual Analysis arXiv:2607.17664
✗ Failed on benchmark 2026

Contractive Latent Observer

Replace recurrence or nearest-neighbour analogue lookup with a learned delay-coordinate observer that continuously corrects a latent state using the current observation. Constrain the observer's closed-loop Jacobian or linear state matrix to have spectral radius below one, so prediction error contracts geometrically and required burn-in grows logarithmically with target accuracy.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: Contraction versus Recurrence: An Exponential Separation in Observation-Based Prediction of Deterministic Dynamics arXiv:2607.14885
✓✓ Beats tuned baseline 2026

Low-rank one-shot horizon predictor

Replace an autoregressive rollout of a learned dynamical model with a branch-trunk factorization that predicts all future steps simultaneously. The branch network encodes the future action sequence, while the trunk network encodes the current state and query coordinates; their inner products produce the complete horizon. This removes repeated state updates during inference and gives a compact differentiable model for planning.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: Model predictive control for laser thermal processing: operator learning, closed-loop validation, and out-of-distribution analysis arXiv:2607.13289
△ Mechanism confirmed, baseline not beaten 2026

Nilpotent BGG Neural Complex

Replace an unconstrained stack of learned vector-field or tensor-field maps by a short neural complex whose fixed differential operators satisfy D_{k+1}D_k=0. The network predicts potentials or quotient representatives, making curl-of-gradient, divergence-of-curl, compatibility, and gauge constraints exact rather than penalty-based.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: The Bernstein-Gelfand-Gelfand (BGG) Construction: Algebra, Geometry, and Analysis; Part I arXiv:2607.10662
✓ Mechanism works 2026

Transport-Signature Density Decoder

Replace a pixelwise density decoder with a decoder that predicts coefficients of a transport potential relative to a fixed reference density. The reconstructed density is the pushforward of the reference measure through a differentiable transport map, so positivity and total mass are structural properties rather than learned penalties.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: Structure-Preserving Reduced-Order Modeling via Low-Rank Transport Signatures arXiv:2607.01696
△ Mechanism confirmed, baseline not beaten 2026

Accumulator-Carrying Picard ResNet

Build a residual module whose state explicitly contains both a persistent context representation and an accumulator. Each residual branch computes one learned correction and adds it to the accumulator, instead of forcing every layer to represent the complete output from scratch. This provides a concrete solver-like architecture for high-dimensional regression and iterative latent prediction.

Useful7/10
Difficulty4/10
Novelty5/10
Paper: Residual neural networks overcome the curse of dimensionality for semilinear heat equations arXiv:2609.03626
△ Mechanism confirmed, baseline not beaten 2026

Piola-Conditioned Fixed-Reference Neural Operator

Build a geometry-conditioned neural operator on a single reference mesh instead of remeshing or changing the network discretization for every domain shape. Transport vector-valued surface features with a contravariant surface Piola map, and feed the network geometry-dependent pulled-back quantities. This should make the architecture stable across shape changes and allow batching many geometries with identical tensor shapes.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Shape Holomorphy and Sparse Approximation of the Maxwell Electric Field Integral Operator arXiv:2609.00466
✓✓ Beats tuned baseline 2026

Intrinsic Tangent-Projected Point-Cloud Layer

Modify a point-cloud message-passing or neural-operator layer so that scalar gradients, vector features, and vector-to-vector interactions are computed only in the estimated tangent plane of the surface. Projecting both feature values and derivative directions prevents the network from using arbitrary ambient-space normal directions and should improve transfer across differently embedded but intrinsically similar surfaces.

Useful7/10
Difficulty4/10
Novelty5/10
Paper: Solving the Incompressible Navier-Stokes Equations on Oriented Curved Surfaces Discretized by Point Clouds arXiv:2609.00216
✗ Failed on benchmark 2026

Rankine–Hugoniot Front Tokens

Augment a 1D neural operator or transformer with explicit tokens for detected discontinuities. Advance each front analytically using the local Rankine–Hugoniot speed and train the network only to reconstruct smooth regions and the residual caused by source terms and grid resolution.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Physics-informed token transformer methodology for nonlinear balance laws. I. Schwarzschild--Burgers fluid flows arXiv:2607.23143
△ Mechanism confirmed, baseline not beaten 2026

Nullspace-coordinate constrained operator blocks

Build a neural operator from frozen ambient mechanism blocks and a geometry-specific algebraic constraint adapter. The adapter parameterizes all outputs in the affine set satisfying sampled linear constraints exactly, so the network never produces boundary-violating states and does not require a penalty coefficient or post-step projection.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Geometry-aware LegONet for PDE Learning on Arbitrary Domains arXiv:2607.23069
△ Mechanism confirmed, baseline not beaten 2026

Conservative Parallel-Edge Decomposition

Represent a multi-input interaction by several single-input edge channels and enforce conservation only after their signed contributions are summed at the vertices. This provides a neural architecture for composite interactions in which different channels have different drivers, while preventing the node update from inventing or destroying net internal flow.

Useful7/10
Difficulty4/10
Novelty5/10
Paper: Multi-Domain Graph-Based Modeling of Energy Systems with Applications to Lithium-Ion Batteries arXiv:2608.30157