Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✓✓ Beats tuned baseline 2026

Adaptive Physics-Lifted Koopman State Space

Replace a purely nonlinear recurrent transition with a learned observable map followed by an explicitly linear latent evolution model. Include the original latent state and a small set of nonlinear observables, and update the linear transition online with forgetting-factor recursive least squares when the environment or task dynamics change.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: Physics-based Online Adaptive Koopman Model Predictive Attitude Control for Combined Spacecraft with Dynamic Uncertainties arXiv:2609.02534
✗ Mechanism failed 2026

Consensus-Corrected Topology-Invariant GNN

Replace ordinary topology-sensitive message passing with scalar-gated aggregation followed by an explicit correction that aligns local node states with a graph-wide consensus component. The correction should make node embeddings less sensitive to line or edge removals while preserving local information needed for prediction. This is suitable for graph neural networks and graph-based world models exposed to changing graph sizes or sparsity patterns.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: UNION: A Unified AC-OPF Framework for Topology-Varying Real-Time Grid Operation arXiv:2608.25784
✗ Failed on benchmark 2026

Conditioned Irregular-Delay State Encoder

Replace uniformly spaced history taps in a neural state-space encoder with a fixed or learned set of non-uniform delays. Regularize the resulting delay-observation matrix to have a large smallest singular value, which makes latent-state reconstruction less sensitive to irregular timestamps and observation noise. This is directly applicable to event-based data, missing timestamps, and systems with multiple time scales.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Stable Takens' Embedding Theorem for Non-Uniformly-Sampled Linear Systems arXiv:2608.14001
△ Mechanism confirmed, baseline not beaten 2026

Stable Modal State-Space Bottleneck

Insert a continuous spatial trunk and a pole-constrained modal state-space branch into a spatiotemporal predictor. The model represents a field as a sum of learned spatial modes and exponentially evolving modal coordinates, so long-horizon behavior is controlled by explicit poles rather than by an unconstrained recurrent transition matrix.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Learning Structural Eigenmodes with Modal Operator Network (ModalONet) arXiv:2607.28926
✗ Failed on benchmark 2026

Contractive Latent Observer

Replace recurrence or nearest-neighbour analogue lookup with a learned delay-coordinate observer that continuously corrects a latent state using the current observation. Constrain the observer's closed-loop Jacobian or linear state matrix to have spectral radius below one, so prediction error contracts geometrically and required burn-in grows logarithmically with target accuracy.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: Contraction versus Recurrence: An Exponential Separation in Observation-Based Prediction of Deterministic Dynamics arXiv:2607.14885
✓ Mechanism works 2026

Transport-Signature Density Decoder

Replace a pixelwise density decoder with a decoder that predicts coefficients of a transport potential relative to a fixed reference density. The reconstructed density is the pushforward of the reference measure through a differentiable transport map, so positivity and total mass are structural properties rather than learned penalties.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: Structure-Preserving Reduced-Order Modeling via Low-Rank Transport Signatures arXiv:2607.01696
△ Mechanism confirmed, baseline not beaten 2026

Quadrature-Whitened Neural Feature Subspace

Freeze a wide neural spatial dictionary, then compress and whiten it using the quadrature mass matrix before solving for output coefficients or latent PDE states. The retained basis removes feature directions that are numerically invisible or nearly dependent under the actual domain discretization, while preserving the represented function space up to the chosen SVD rank.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Evo-GTransNet for Parabolic PDEs: A Fixed-Feature Galerkin Method of Lines with Quadrature-Mass Orthonormalization arXiv:2608.19615
✗ Mechanism failed 2026

Gaussian-mixture kinetic neural solver

Make a neural network predict a positive Gaussian-mixture representation of the distribution function rather than independent values on a momentum grid. Use the mixture parameters inside a differentiable Boltzmann collision operator, so training directly enforces the interaction mechanism and exposes the relaxation spectrum responsible for ballistic-to-hydrodynamic crossover.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Linear response across interaction regimes in two-dimensional ferromagnets arXiv:2608.14477
✓✓ Beats tuned baseline 2026

Divergence-Free Spherical Kernel Layer

Build a kernel aggregation layer whose output is a tangent vector field on the unit sphere and whose surface divergence is identically zero by construction. For each source point, use a matrix kernel obtained by applying a surface-rotated gradient in the query variable to a scalar zonal kernel; this is a differential-form version of the paper's matrix-valued construction. The layer can replace attention or message passing when the target dynamics are incompressible, such as spherical fluid…

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Divergence-free interpolation of tangential vector fields via matrix-valued kernels arXiv:2608.05547
△ Mechanism confirmed, baseline not beaten 2026

Explicit Multi-Timescale Memory Bank

Use a bank of damped rotational state channels with a deliberately spread decay spectrum, allowing one recurrent layer to represent short, medium, and long temporal dependencies without relying on a single learned spectral radius. Concatenate the channels and train a readout or downstream nonlinear head to select the appropriate memory timescale.

Useful7/10
Difficulty4/10
Novelty7/10
Paper: Lindblad-Inspired Multi-Timescale Reservoir Computing with Separable Rotation and Dissipation arXiv:2608.04028
△ Mechanism confirmed, baseline not beaten 2026

Non-Gaussian Perron–Frobenius Latent Filter

Replace Gaussian covariance propagation in a neural state-space model with a finite Perron–Frobenius operator acting on coefficients of a learned density basis. A neural encoder maps observations to latent states, while an eDMD-derived matrix transports the full coefficient vector and supports multimodal or skewed uncertainty. This creates a cheap deterministic uncertainty layer that can be rolled forward for long horizons without repeatedly sampling particles.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: An extended Perron-Frobenius operator filter for nonlinear state estimation arXiv:2607.26632
△ Mechanism confirmed, baseline not beaten 2026

Delay-Aware Frequency-Preserving Recurrent Coupling

For coupled recurrent or state-space modules that represent oscillatory or periodic signals, explicitly account for communication or attention delay in the characteristic equation. Tune the coupling gain or add a phase-lead compensator so that the desired latent frequency remains a closed-loop mode instead of being shifted by small delays.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: How network perturbations distort agreement trajectories in LTI multi-agent systems arXiv:2607.18913
△ Mechanism confirmed, baseline not beaten 2026

Noise-Triggered Latent Rank Adaptation

Use the recursive errors-in-variables subspace spectrum as a controller for the width of a latent SSM rather than fixing the state dimension in advance. Neurons or state channels are added when corrected covariance eigenvalues rise above the noise floor and pruned when they remain below it, producing a model-order-adaptive recurrent architecture for nonstationary streams.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: A recursive subspace based method for errors-in-variables model identification of time-varying systems arXiv:2607.17065
✗ Mechanism failed 2026

Volterra BBP multi-timescale recurrent initialization

Initialize a recurrent or state-space transition matrix with weak Wigner noise plus a shared cumulative-sum correlation structure. Increasing the correlation strength produces recurrent eigenmodes one at a time at analytically predicted BBP thresholds, yielding a controllable hierarchy of short- and long-memory modes. The matrix should then be globally rescaled or constrained so that all active modes remain inside the desired stability radius.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: A Cascade of Volterra-Operator BBP Transitions in a Correlated Wigner Matrix arXiv:2607.10503
✓✓ Beats tuned baseline 2026

Cyclic Multirate Neural State Space

Replace interpolation of heterogeneous sensor streams by a phase-indexed recurrent or state-space network with period M, where M is the least common multiple of the sensor sampling periods. The network applies a distinct transition for each phase while using a fixed cyclic phase update, preserving timing structure and allowing missing observations to enter only when their phase-specific sensor is available.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Cyclic Reformulation-Based Identification and Polytopic Uncertainty Modeling for Multirate Systems arXiv:2607.09194
✗ Mechanism failed 2026

DMDc-Initialized Marginal-Stable Neural SSM

Replace the unconstrained transition of a recurrent or state-space neural network with a DMDc-initialized linear latent transition plus a learned nonlinear residual. Estimate the transition from a short warm-up dataset using Hankel delay coordinates, retain eigenmodes with decay rates near the unit circle for long-term memory, and let the neural residual model dynamics not explained by the linear backbone. This should make long-horizon prediction and slowly varying signals easier to learn while…

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Koopman Spectral Analysis of Lithium-Ion Battery Dynamics: State of Charge as a Marginally Stable Observable arXiv:2607.07594
Unverified 2026

Integrable Elliptic Memory Cell

Replace an unconstrained recurrent transition with block-diagonal planar rotations whose angles are learned or conditioned on a slowly varying context variable. The resulting hidden-state norm and each two-dimensional block energy are exactly invariant in the ideal recurrence, preventing exploding or vanishing recurrent dynamics while retaining phase information over long horizons.

Useful6/10
Difficulty4/10
Novelty4/10
Paper: Outer Contact Billiards arXiv:2608.19393
Unverified 2026

Surrogate-guided topology search for nonlinear reservoirs

Search sparse reservoir wiring in graph space rather than repeatedly testing every candidate with its full nonlinear dynamics. Use graph descriptors to predict validation accuracy and nonlinear feature selectivity, then spend exact simulations on candidates with high predicted performance or high surrogate uncertainty.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Graph-theoretic design of lasing networks for physical vision arXiv:2608.13097
Unverified 2026

Complex-stretched resonance layer

Insert a fixed or learnable complex coordinate stretch outside the region where a neural operator models the physical interaction, so outgoing waves are damped and resonant states become ordinary discrete eigenmodes on a finite grid. Train the network with eigenvalue or resolvent losses computed after the stretch, while preserving the physical field in the interior region.

Useful5/10
Difficulty6/10
Novelty7/10
Paper: Dirac resonances as non-self-adjoint eigenvalues arXiv:2607.26166