Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✓✓ Beats tuned baseline 2026

Conservative Flux Neural Operator

Replace pointwise prediction of the next field with prediction of a learned flux followed by a discrete divergence. Combine Fourier spatial mixing with a causal temporal kernel over the recent resolved-history slab, so the model learns finite-memory closure effects while preserving local conservation exactly under periodic or compatible boundary conditions. The architecture should reduce spurious mass drift and improve autoregressive rollout stability on coarse-grained PDE data.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: Flux-form spatiotemporal neural operators for coarse-grained dynamics of multiscale PDEs arXiv:2608.18148
✓✓ Beats tuned baseline 2026

Equivariant Variational Field Network

Represent a scalar energy or free-energy functional of a three-dimensional neural field using translation- and rotation-equivariant convolutions, and produce the field prediction by minimizing the total functional rather than by a direct decoder. The same functional can then generate equilibrium states, forces, and response observables under new external fields, resolutions, and system sizes.

Useful8/10
Difficulty7/10
Novelty6/10
Paper: Equivariant learning of a transferable three-dimensional classical density functional arXiv:2608.13506
✓✓ Beats tuned baseline 2026

Nonlinearity-Subtracted Latent State-Space Model

Build a latent continuous-time neural model with dynamics \(\dot{z}=Az+f_\phi(z)\), where \(f_\phi\) is known, separately computed, or frozen, and \(A\) is learned exclusively from the derivative residual after subtracting \(f_\phi(z)\). Parameterize \(A\) with a truncated SVD or low-rank factorization so its eigenvalues directly predict local stability and long-horizon growth.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Data-driven linear analysis of dynamical systems via nonlinearity-subtracted dynamic mode decomposition arXiv:2608.13373
✓✓ Beats tuned baseline 2026

Inverse-Gain Structured Privileged Distillation

Replace direct action imitation with a causal recurrent estimator of the inverse input gain. The neural network predicts the latent quantity needed by the expert controller, and a fixed algebraic wrapper converts that prediction into an action using the measured state difference and tracking error, thereby removing the additive disturbance exactly under the sampled timing model.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: From Privileged Control to Deployable Adaptation:Fusing Mechanism-Guided Task Reduction with Learned Behavior arXiv:2608.10453
✓✓ Beats tuned baseline 2026

Power-Balanced Modular Neural Block

Represent each neural module as a Hamiltonian storage system and connect modules through a state-dependent skew or Dirac interconnection instead of arbitrary residual additions. The coupling may change with the hidden state, but its internal power contribution cancels exactly, so total stored energy is controlled only by external inputs and explicitly added dissipation.

Useful8/10
Difficulty5/10
Novelty5/10
Paper: Port-Hamiltonian modelling of coupled rigid/flexible multibody systems arXiv:2608.05143
✓✓ Beats tuned baseline 2026

Stable Rotating-Memory State Space

Replace an unconstrained recurrent transition with a decaying symmetric memory operator plus a skew-symmetric rotational operator. The skew component creates phase-shifted cross-channel memory and can represent oscillatory or circulatory temporal dependencies without requiring eigenvalues with large positive real parts.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: Memory with Onsager-Casimir symmetry: Rotating particle in a viscoelastic fluid arXiv:2608.00344
✓✓ Beats tuned baseline 2026

Backward-Equivalent Quotient GNN

Partition graph nodes into backward-equivalent classes and run message passing on the K-node quotient graph instead of the original N-node graph. If every node in a class receives the same aggregate message from every source class and shares the same local update map, class-constant node representations remain class-constant at every layer, making the quotient computation exactly equivalent to the full GNN on that invariant subspace.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: Aggregation of Evolutionary Game Dynamics on Large-Scale Weighted Networks arXiv:2607.18776
✓✓ Beats tuned baseline 2026

Conservative Chapman–Enskog Neural Layer

Replace an unconstrained recurrent hidden-state update by a fast redistribution state with a dissipative Jacobian and a slow conserved state. The network computes an equilibrium state and a first-order pseudoinverse response correction, transferring the paper’s separation between local relaxation and macroscopic transport into a stable recurrent or state-space layer.

Useful8/10
Difficulty7/10
Novelty7/10
Paper: Richards' equation as a hydrodynamic limit: Chapman--Enskog reduction of the continuum kinetic equation for unsaturated soil water arXiv:2607.17358
✓✓ Beats tuned baseline 2026

Kesten–Stigum Attenuated Message Passing

Replace uniform graph-convolution aggregation with a distance-aware message transform whose strength decays as \(\gamma^k\). At hop \(k\), transform the learned local evidence with \(2\operatorname{artanh}(\gamma^k z)\) before summation, so distant nodes have a provably shrinking influence window rather than accumulating unbounded noisy evidence.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: The Value of Depth in Message Passing on Sparse Graphs: A Kesten-Stigum Dichotomy arXiv:2607.16676
✓✓ Beats tuned baseline 2026

Spectral pinning of neural modules

Represent communicating layers, experts, or distributed workers as nodes of a weighted graph and apply strong corrective updates only to a small pinned subset. Select pins by the increase they produce in the grounded Laplacian smallest eigenvalue, because this spectral gap predicts the decay rate of representation disagreement.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Event-Triggered Pinning Impulsive Control of Complex Networks with Actuation Delays: Stability Analysis and Zeno-Free Conditions arXiv:2608.24074
✓✓ Beats tuned baseline 2026

Fourier-Mode Stability Shaping

Use a cyclic forward-neighbor recurrent or state-space layer and regularize its coupling so selected discrete Fourier modes are contracting while task-critical modes remain weakly damped. The paper's exact mode factors make instability falsifiable: a mode becomes unstable when its scalar factor changes sign, producing a measurable transition rather than a vague smoothness prior.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: Existence and Stability of Dancing Equilibria in Asymmetric Kuramoto Networks arXiv:2608.29630
✓✓ Beats tuned baseline 2026

Tiny Local Recurrence with Adaptive Computation

Replace a stack of independently parameterized residual or MLP blocks with a small latent grid or vector repeatedly updated by one shared transition rule. Let the number of updates depend on the current latent state, so easy examples terminate early while hard examples receive more computation, potentially improving parameter efficiency and extrapolation.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Emergent Models: Intelligence from Tiny Substrates arXiv:2608.14019
✓✓ Beats tuned baseline 2026

Two-Channel Fractal Renormalization Network

Replace ordinary depth-wise feature propagation by a ternary hierarchical block that recursively aggregates three child representations while maintaining separate neutral and defect channels. The block is initialized from the Sierpinski six-vertex recursion, then optionally learns a bounded correction. The neutral channel preserves the paper's cubic mixing law, while the defect channel provides a controlled route for long-range and nonlocal interactions.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Fractal deconfinement and confinement in Sierpinski ice arXiv:2608.02741
✓✓ Beats tuned baseline 2026

Symmetry-Preserving Flow Layer

Construct hidden dynamics from permutation-equivariant vector fields and impose antisymmetry through an explicit antisymmetrizing readout. This prevents optimization from learning multiple equivalent copies of the same configuration and makes forbidden symmetry violations exactly zero, rather than merely penalizing them. The design applies to set models, particle systems, graph networks, and architectures handling unordered tokens.

Useful7/10
Difficulty5/10
Novelty4/10
Paper: Spindrift: Learning quantum degeneracy from thermal purity in restricted path integral Monte Carlo arXiv:2607.29590
✓✓ Beats tuned baseline 2026

Noisy Scrambling-Front Network

Construct a residual sequence or depth network whose nonnegative influence density follows a discretized noisy Fisher-KPP equation: local influence diffuses, grows when small, saturates at a finite carrying capacity, and receives state-dependent noise. Use this density to gate ordinary feature updates rather than relying only on unconstrained residual additions. The mechanism predicts a measurable propagation speed and an instability boundary, allowing the architecture to be falsified…

Useful7/10
Difficulty6/10
Novelty8/10
Paper: Effective Field Theory of Operator Scrambling from Strong-to-Weak Symmetry Breaking arXiv:2607.24925
✓✓ Beats tuned baseline 2026

Cyclic Momentum Latents

Add a learnable cyclic-coordinate mechanism to latent dynamics so that selected latent coordinates do not enter the Hamiltonian and their conjugate momenta become conserved. This provides an explicit dimensionality-discovery and invariance bias, encouraging the model to represent nuisance or symmetry directions compactly instead of spending independent dynamics capacity on them.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Generating Physically Plausible Parachute Dynamics with Deep Generative Modeling arXiv:2607.12143
✓✓ Beats tuned baseline 2026

Local Characteristic Residual Gating

Transform local neural residuals into the Ripa model's characteristic coordinates before spatial aggregation, apply a mode-dependent gate based on neighboring characteristic jumps, and transform back. This lets the model damp oscillatory acoustic or equilibrium-mode corrections near discontinuities without globally smoothing every feature.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Fifth-Order Well-Balanced Path-Conservative A-WENO Scheme for the Ripa Model arXiv:2607.09293
✓✓ Beats tuned baseline 2026

Agnostic Geometry-Prior Mixer

Train an unconstrained branch and a geometry-aware branch in parallel, then learn how much to trust the analytic branch. This preserves the benefit of explicit geometry on correctly specified tasks while allowing the model to ignore a misleading or irrelevant prior.

Useful6/10
Difficulty3/10
Novelty7/10
Paper: Geometry-Aware R-Structured Kolmogorov-Arnold Networks arXiv:2607.01449
✓✓ Beats tuned baseline 2026

Pole-safe rational neural layer

Replace an unconstrained feature vector entering a rational or resolvent-like neural operator by a polynomial feature whose first nonzero Taylor coefficient lies in a pole-safe subspace. For a pole of order m, the simplest guaranteed construction is psi(z)=(z-beta)^m v, which makes Q(z)psi(z) bounded even when Q(z) diverges. For lower-order cancellation, solve linear constraints among Taylor coefficients of psi so that all negative Laurent powers vanish.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Construction of Pole Cancellation Functions at Ordinary Poles of Operator-Valued Functions arXiv:2607.00097
✓✓ Beats tuned baseline 2026

Reversible Matrix Cluster Layer

Construct a latent layer whose node states are small positive-definite matrices and whose local updates follow a weighted cluster exchange relation rather than an unconstrained affine map. The update is reversible when the old state is retained, while noncommuting matrix products preserve relational structure that scalar cluster variables cannot represent.

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Noncommutative Cluster Varieties and Moduli Spaces of Local Systems arXiv:2608.27284
✓✓ Beats tuned baseline 2026

Minkowski-Additive Convex Latents

Store a convex object as a direction-indexed vertex tuple and implement composition of objects through componentwise Minkowski addition and nonnegative scaling. This creates a structured residual or compositional layer where convexification is nonexpansive, making perturbation amplification controllable and avoiding repeated generic geometric optimization.

Useful6/10
Difficulty4/10
Novelty8/10
Paper: Galerkin approximations to the space of convex bodies by polytopes in nondegenerate V-representation arXiv:2608.26615
✓✓ Beats tuned baseline 2026

Cyclic Lie-Bracket Residual Block

Replace one deterministic residual update with a short cyclic composition of learned vector fields evaluated for randomized, short run times. Because finite compositions of noncommuting flows generate directional-derivative and Lie-bracket terms, changing the cycle order gives the network an explicit, low-cost way to learn drift directions that are unavailable from the individual vector fields alone.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Diffusion limits of cyclic finite-velocity random motions along vector fields arXiv:2608.22514