Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

△ Mechanism confirmed, baseline not beaten 2026

Stable Modal State-Space Bottleneck

Insert a continuous spatial trunk and a pole-constrained modal state-space branch into a spatiotemporal predictor. The model represents a field as a sum of learned spatial modes and exponentially evolving modal coordinates, so long-horizon behavior is controlled by explicit poles rather than by an unconstrained recurrent transition matrix.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Learning Structural Eigenmodes with Modal Operator Network (ModalONet) arXiv:2607.28926
△ Mechanism confirmed, baseline not beaten 2026

Disorder-Nonreciprocal Oscillatory RNN

Replace a conventional recurrent transition by two coupled hidden channels with equal-and-opposite cross-couplings and a controllable disorder scale. The antisymmetric coupling produces complex recurrent eigenmodes, providing oscillatory memory rather than purely monotone decay, while the disorder parameter controls the real part of the eigenvalues and therefore the stability margin.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: Disorder induced time crystal in athermal random field Ising model with non-reciprocal interactions arXiv:2607.28781
✓✓ Beats tuned baseline 2026

Lattice Error-Feedback Residual Blocks

Replace full-state quantized write-back in a deep low-bit residual stack with quantized increment error feedback. The residual branch quantizes the proposed increment after adding the previous carry, while the carry stores the exact discrepancy; this makes the total error telescope instead of accumulating approximately once per layer.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: When Can Depth Replace Precision? A Resource Theory of Quantized Neural Computation arXiv:2607.23390
△ Mechanism confirmed, baseline not beaten 2026

Belief-Intersection Forcing Shield

Add a discrete-event safety shield between a partially observed neural policy and the environment. The policy proposes a forcing action, but the shield permits it only when the same decision is safe for every latent plant state compatible with the current observation; otherwise it returns a certified inconsistency or a conservative fallback. This converts forcing consistency into an implementable robust action-selection rule rather than trusting a single estimated hidden state.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Supervisory Control with Event Forcing Under Partial Observation arXiv:2607.21040
✓✓ Beats tuned baseline 2026

Backward-Equivalent Quotient GNN

Partition graph nodes into backward-equivalent classes and run message passing on the K-node quotient graph instead of the original N-node graph. If every node in a class receives the same aggregate message from every source class and shares the same local update map, class-constant node representations remain class-constant at every layer, making the quotient computation exactly equivalent to the full GNN on that invariant subspace.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: Aggregation of Evolutionary Game Dynamics on Large-Scale Weighted Networks arXiv:2607.18776
△ Mechanism confirmed, baseline not beaten 2026

Edge-Supported Polynomial State Space

Replace a complete tensor/Kronecker polynomial lift of a graph dynamical system with observables selected only from the support of the interaction graph. The lifted state can then be propagated by a sparse structured linear operator, while the first omitted degree is treated as an explicit residual or learned closure. This gives a graph-aware polynomial state-space layer for neural ODEs, graph RNNs, and world models.

Useful8/10
Difficulty5/10
Novelty8/10
Paper: Graph-Induced Tensor Liftings for Networked SEIR Models: Dimensional Reduction and Residual Analysis arXiv:2607.17664
✓✓ Beats tuned baseline 2026

Conservative Chapman–Enskog Neural Layer

Replace an unconstrained recurrent hidden-state update by a fast redistribution state with a dissipative Jacobian and a slow conserved state. The network computes an equilibrium state and a first-order pseudoinverse response correction, transferring the paper’s separation between local relaxation and macroscopic transport into a stable recurrent or state-space layer.

Useful8/10
Difficulty7/10
Novelty7/10
Paper: Richards' equation as a hydrodynamic limit: Chapman--Enskog reduction of the continuum kinetic equation for unsaturated soil water arXiv:2607.17358
△ Mechanism confirmed, baseline not beaten 2026

q-Fractional Memory State-Space Layer

Replace the uniform or power-law convolution in a recurrent or state-space layer by a Gaussian q-binomial fractional kernel with learnable order alpha and deformation q. The parameter q controls a concrete memory-localization transition: q close to 1 gives classical fractional power-law memory, whereas q<1 produces exponentially localized memory and should reduce long-horizon gradient interference and truncation cost.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Maps of q-deformed fractional order: From circle to cardioid via crescent arXiv:2607.15833
✗ Failed on benchmark 2026

Contractive Latent Observer

Replace recurrence or nearest-neighbour analogue lookup with a learned delay-coordinate observer that continuously corrects a latent state using the current observation. Constrain the observer's closed-loop Jacobian or linear state matrix to have spectral radius below one, so prediction error contracts geometrically and required burn-in grows logarithmically with target accuracy.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: Contraction versus Recurrence: An Exponential Separation in Observation-Based Prediction of Deterministic Dynamics arXiv:2607.14885
△ Mechanism confirmed, baseline not beaten 2026

High-Order Flat-Band Residual Dynamics

Replace standard nearest-neighbor residual or recurrent mixing with a learned multi-range shift operator whose coefficients cancel low-order derivatives of its Fourier symbol at a selected momentum. This creates slow modes with dispersion of order W, which should preserve low-frequency information over longer horizons while retaining an explicitly measurable spectral signature.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: From stable periodic orbits to many-body chaos: doubly tunable prethermalization via engineering of an emergent band structure arXiv:2607.12355
✗ Failed on benchmark 2026

Small-gain certified modular network

Partition a neural network into independently trained or independently monitored modules and constrain their cross-module interaction gain using a compositional contraction certificate. This enables stable deep modular MLPs, graph blocks, or recurrent modules without estimating the full network Jacobian, while providing an explicit coupling threshold for when the architecture loses contraction.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Contraction Certification from Streaming Data: Wasserstein Robustness and Compositional Stability for Interconnected Nonlinear System arXiv:2607.11982
△ Mechanism confirmed, baseline not beaten 2026

Certified contraction implicit layer

Replace a deep feed-forward block by the fixed point z=phi(Wz+Vx+b), with the recurrent weight W constrained so that the fixed point is unique for every input. The same condition makes forward fixed-point iteration stable and makes implicit differentiation well-conditioned, allowing depth-independent memory usage while providing a measurable spectral failure boundary.

Useful8/10
Difficulty5/10
Novelty4/10
Paper: Implicit Neural Networks as Static Controllers: Certificates and Performance Separation arXiv:2607.11122
✗ Failed on benchmark 2026

Topology-Aware Streaming Jacobian Monitor

For a recurrent or graph neural network with known local connectivity, estimate each node's local Jacobian row using only graph neighbors rather than all hidden coordinates. Use the resulting sparse Jacobian both to compute a contraction certificate and to regularize training toward dynamically local interactions, reducing estimator variance and the number of samples required for reliable stability decisions.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Streaming Contraction Certificates for Nonlinear Networks: Topology-Aware Data Sufficiency with Partial Observation arXiv:2607.10893
△ Mechanism confirmed, baseline not beaten 2026

Nilpotent BGG Neural Complex

Replace an unconstrained stack of learned vector-field or tensor-field maps by a short neural complex whose fixed differential operators satisfy D_{k+1}D_k=0. The network predicts potentials or quotient representatives, making curl-of-gradient, divergence-of-curl, compatibility, and gauge constraints exact rather than penalty-based.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: The Bernstein-Gelfand-Gelfand (BGG) Construction: Algebra, Geometry, and Analysis; Part I arXiv:2607.10662
△ Mechanism confirmed, baseline not beaten 2026

Certified Bi-Lipschitz Recurrent Cell

Replace an unconstrained RNN or state-space layer with an implicit recurrent cell whose nonlinear algebraic loop is well posed and whose forward dynamics are contracting and strongly input-output monotone. The same certificate guarantees a causal inverse with bounded gain, so sequence predictions should be insensitive to initial-state mismatch while remaining responsive to input perturbations.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Robustly Invertible Nonlinear Dynamics and the BiLipREN: From Inversion-Based Control to Generative Trajectory Modelling arXiv:2607.10026
✗ Failed on benchmark 2026

Balanced State-Order Compression

Compress each hidden layer by retaining directions that are simultaneously reachable from the observed input distribution and observable at the network output. Unlike PCA or SVD, the retained subspace is weighted by downstream task sensitivity, so high-variance but output-irrelevant directions can be removed while low-variance predictive directions are preserved.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Empirical Minimal-Realisation Compression of Deep Neural Networks via Controllability-Observability Tests arXiv:2607.05457
✗ Mechanism failed 2026

Dendritic LMS State for Spiking ICL

Give a single spiking layer a persistent vector-valued apical compartment that stores the current online linear predictor for the task. On each labeled context pair, its subthreshold state performs a leaky LMS update; on the query, the state is read without updating, allowing in-context adaptation without attention or inference-time synaptic plasticity.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Dendritic In-Context Learning in a Single-Layer Spiking Neural Network arXiv:2607.02283
✗ Mechanism failed 2026

Correlated Long-Range Residual Mixer

Replace an iid local or randomly sparse residual mixer with a distance-decaying long-range operator whose edge magnitudes are correlated through a shared latent Gaussian field. The paper predicts that these correlations qualitatively change low-energy spectral scaling and increase multiscale information propagation relative to iid long-range weights. Apply the operator as a spectrally normalized residual block so that the benefit comes from correlated scale coverage rather than uncontrolled…

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Effective Hamiltonian description on monitored Majorana chains: correlated power-law hoppings and unconventional entanglement scaling arXiv:2609.04091
△ Mechanism confirmed, baseline not beaten 2026

Accumulator-Carrying Picard ResNet

Build a residual module whose state explicitly contains both a persistent context representation and an accumulator. Each residual branch computes one learned correction and adds it to the accumulator, instead of forcing every layer to represent the complete output from scratch. This provides a concrete solver-like architecture for high-dimensional regression and iterative latent prediction.

Useful7/10
Difficulty4/10
Novelty5/10
Paper: Residual neural networks overcome the curse of dimensionality for semilinear heat equations arXiv:2609.03626
△ Mechanism confirmed, baseline not beaten 2026

Degree-Calibrated Stable Residual Flow

Construct a continuous-depth or recurrent residual block with a prescribed polynomial Lyapunov decay near its equilibrium. The architecture combines a fixed radial stabilizer with a learned component that is constrained to have zero radial projection, allowing slow algebraic transients and long memory while preventing asymptotic hidden-state growth.

Useful7/10
Difficulty6/10
Novelty8/10
Paper: On a Gradation for Asymptotic Stability arXiv:2609.03120
✗ Failed on benchmark 2026

Positive-real rational resolvent mixer

Replace an unconstrained token mixer or feed-forward residual map with a feature-conditioned rational operator whose transfer matrix is positive real on the positive orthant. Apply it through a resolvent, rather than an additive residual, so that the accretivity certificate yields a nonexpansive implicit update and suppresses activation explosions.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: The classes of bivariate Schur and Herglotz matrix-valued rational functions: realizations, symmetrizations, and related determinantal representations arXiv:2609.03054
✓✓ Beats tuned baseline 2026

Persistent Relational Memory

Give every possible pairwise relation its own recurrent latent state, stored in a dictionary keyed by stable node identities, instead of discarding the state whenever the edge is absent from the current graph. At each timestep, active edges retrieve their old state, update it with the current pair features and neighborhood messages, and write it back after message passing. This should improve predictions in dynamic graphs with intermittent interactions and reduce the burden on node states to…

Useful7/10
Difficulty5/10
Novelty6/10
Paper: TRACE: Spatiotemporal Contact Memory Graph Network Simulator for Granular Dynamics arXiv:2609.02991
△ Mechanism confirmed, baseline not beaten 2026

Envelope-Max Neural Operator

Build each one-step operator as a maximum over a compact set of learned or discretized action branches, with a branch-dependent penalty. This directly imports the envelope structure used for nonlinear semigroups and gives a neural architecture suited to HJB equations, robust prediction, and stochastic control under model uncertainty.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Neural operators approximate strongly continuous convex monotone semigroups arXiv:2609.02727
△ Mechanism confirmed, baseline not beaten 2026

Spectral Basin Allocation for Multimodal Neural Memories

Use several phase-locked states as distinct attractors of one recurrent network and shape their basin asymmetry through the phase-delay composite spectrum. This creates a controllable associative-memory architecture in which a desired memory receives a larger basin without adding a separate classifier or explicit nearest-neighbor lookup.

Useful7/10
Difficulty7/10
Novelty8/10
Paper: Phase-delays shape multistability and basin sizes in Kuramoto networks: analytical estimates from network structure arXiv:2609.02047