Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✓✓ Beats tuned baseline 2026

Adaptive Physics-Lifted Koopman State Space

Replace a purely nonlinear recurrent transition with a learned observable map followed by an explicitly linear latent evolution model. Include the original latent state and a small set of nonlinear observables, and update the linear transition online with forgetting-factor recursive least squares when the environment or task dynamics change.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: Physics-based Online Adaptive Koopman Model Predictive Attitude Control for Combined Spacecraft with Dynamic Uncertainties arXiv:2609.02534
✓✓ Beats tuned baseline 2026

Resolution-normalized Hilbert dictionary

Build a shallow neural model whose input at every discretization level is embedded into a common Hilbert space with uniformly bounded norm, and constrain every neuron parameter in the corresponding dual norm. The statistical complexity then depends on the Hilbert norm bound rather than the number of retained coordinates, allowing one model design to operate across increasingly fine measurements.

Useful8/10
Difficulty4/10
Novelty7/10
Paper: Resolution-Consistent Greedy Neural Approximation on Infinite-Dimensional Spaces arXiv:2608.20812
✓✓ Beats tuned baseline 2026

Conservative Flux Neural Operator

Replace pointwise prediction of the next field with prediction of a learned flux followed by a discrete divergence. Combine Fourier spatial mixing with a causal temporal kernel over the recent resolved-history slab, so the model learns finite-memory closure effects while preserving local conservation exactly under periodic or compatible boundary conditions. The architecture should reduce spurious mass drift and improve autoregressive rollout stability on coarse-grained PDE data.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: Flux-form spatiotemporal neural operators for coarse-grained dynamics of multiscale PDEs arXiv:2608.18148
✓✓ Beats tuned baseline 2026

Equivariant Variational Field Network

Represent a scalar energy or free-energy functional of a three-dimensional neural field using translation- and rotation-equivariant convolutions, and produce the field prediction by minimizing the total functional rather than by a direct decoder. The same functional can then generate equilibrium states, forces, and response observables under new external fields, resolutions, and system sizes.

Useful8/10
Difficulty7/10
Novelty6/10
Paper: Equivariant learning of a transferable three-dimensional classical density functional arXiv:2608.13506
✓✓ Beats tuned baseline 2026

Power-Balanced Modular Neural Block

Represent each neural module as a Hamiltonian storage system and connect modules through a state-dependent skew or Dirac interconnection instead of arbitrary residual additions. The coupling may change with the hidden state, but its internal power contribution cancels exactly, so total stored energy is controlled only by external inputs and explicitly added dissipation.

Useful8/10
Difficulty5/10
Novelty5/10
Paper: Port-Hamiltonian modelling of coupled rigid/flexible multibody systems arXiv:2608.05143
✓✓ Beats tuned baseline 2026

Intrinsic Tangent-Projected Point-Cloud Layer

Modify a point-cloud message-passing or neural-operator layer so that scalar gradients, vector features, and vector-to-vector interactions are computed only in the estimated tangent plane of the surface. Projecting both feature values and derivative directions prevents the network from using arbitrary ambient-space normal directions and should improve transfer across differently embedded but intrinsically similar surfaces.

Useful7/10
Difficulty4/10
Novelty5/10
Paper: Solving the Incompressible Navier-Stokes Equations on Oriented Curved Surfaces Discretized by Point Clouds arXiv:2609.00216
✓✓ Beats tuned baseline 2026

Reciprocal-Lattice Gauge-Covariant Bloch Network

Build a Bloch-conditioned neural model whose periodic-factor representation transforms covariantly when the supplied Bloch wavenumber is shifted by a reciprocal lattice vector. Either canonicalize q to the first Brillouin zone or augment training with mathematically paired examples whose outputs differ by the exact phase gauge. This prevents the network from learning inconsistent predictions for physically identical Bloch modes.

Useful7/10
Difficulty4/10
Novelty8/10
Paper: Full-field and Bloch-periodic-factor discretizations: Accuracy and phantom modes arXiv:2608.14348
✓✓ Beats tuned baseline 2026

Displacement-Huber distribution pooling

Replace ordinary Wasserstein or arithmetic pooling of distribution-valued features with a barycenter whose individual quantile displacements are Huberized. Small changes between input distributions remain averaged quadratically, while a corrupted token, expert, graph neighborhood, or augmentation cannot move the pooled distribution arbitrarily far. The module is especially cheap for one-dimensional distributions represented by fixed quantile vectors.

Useful7/10
Difficulty4/10
Novelty7/10
Paper: Huber-Wasserstein barycenters for robust distribution-valued data arXiv:2608.13131
✓✓ Beats tuned baseline 2026

Divergence-Free Spherical Kernel Layer

Build a kernel aggregation layer whose output is a tangent vector field on the unit sphere and whose surface divergence is identically zero by construction. For each source point, use a matrix kernel obtained by applying a surface-rotated gradient in the query variable to a scalar zonal kernel; this is a differential-form version of the paper's matrix-valued construction. The layer can replace attention or message passing when the target dynamics are incompressible, such as spherical fluid…

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Divergence-free interpolation of tangential vector fields via matrix-valued kernels arXiv:2608.05547
✓✓ Beats tuned baseline 2026

Holonomy-Twisted Message Passing

Replace an ordinary graph-neural-network edge message by a message transported through a unitary representation of the edge's fundamental-group label. The layer can distinguish globally different holonomy sectors even when the underlying bundles or ordinary graph topology are identical, while inverse edge labels enforce a Hermitian and unitary consistency constraint.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Beyond $K$-Theory: Geometry and Holomorphy in Hyperbolic Band Theory arXiv:2608.01596
✓✓ Beats tuned baseline 2026

Symmetry-Preserving Flow Layer

Construct hidden dynamics from permutation-equivariant vector fields and impose antisymmetry through an explicit antisymmetrizing readout. This prevents optimization from learning multiple equivalent copies of the same configuration and makes forbidden symmetry violations exactly zero, rather than merely penalizing them. The design applies to set models, particle systems, graph networks, and architectures handling unordered tokens.

Useful7/10
Difficulty5/10
Novelty4/10
Paper: Spindrift: Learning quantum degeneracy from thermal purity in restricted path integral Monte Carlo arXiv:2607.29590
✓✓ Beats tuned baseline 2026

KPZ latent evolution block

Replace an unconstrained recurrent or neural-operator latent transition with a differentiable KPZ cell acting on a spatial latent field. The cell explicitly separates smoothing, nonequilibrium nonlinear steepening, and stochastic forcing, making it suitable for driven dissipative systems and long-horizon roughening that generic networks may fail to reproduce.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Three-Dimensional Kardar--Parisi--Zhang Scaling in Polariton Condensates arXiv:2607.28106
✓✓ Beats tuned baseline 2026

Fermionic circuit message passing

Augment every graph-neural-network edge message with an even commuting channel and a low-dimensional odd anticommuting channel. Contracting odd channels around an edge circuit gives a sign determined by the number of odd edges, while local states with odd incident degree are forced to vanish; this supplies a built-in parity and cycle constraint that ordinary GNNs must learn implicitly.

Useful7/10
Difficulty6/10
Novelty8/10
Paper: Mixed partition functions are exactly the graph parameters of exponentially bounded edge-connection rank arXiv:2607.27198
✓✓ Beats tuned baseline 2026

Incidence-Matrix Structured Action Head

Build a policy or prediction head that scores atomic graph entities once and converts those scores into logits for legal composite actions through an instance-specific incidence matrix. The neural parameters are independent of the number of actions and their composition, so the same model can process graphs with different action counts and compatibility structures without padding every action space to a global maximum.

Useful7/10
Difficulty4/10
Novelty6/10
Paper: A Graph-Based Control Interface for Traffic Signals on Heterogeneous Road Networks arXiv:2607.21831
✓✓ Beats tuned baseline 2026

Cyclic Momentum Latents

Add a learnable cyclic-coordinate mechanism to latent dynamics so that selected latent coordinates do not enter the Hamiltonian and their conjugate momenta become conserved. This provides an explicit dimensionality-discovery and invariance bias, encouraging the model to represent nuisance or symmetry directions compactly instead of spending independent dynamics capacity on them.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Generating Physically Plausible Parachute Dynamics with Deep Generative Modeling arXiv:2607.12143
✓✓ Beats tuned baseline 2026

Conformal-symplectic sandwich layer

Replace an unconstrained one-step transition network with a symmetric damping–symplectic-core–damping composition. The damping strength is one learned scalar rate and is applied through positive exponential diagonal factors, so every step has a known contraction law while the neural core models nonlinear conservative transport.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: CSympNet-ID: conformal-symplectic map learning for linearly damped Hamiltonian systems arXiv:2607.03339
✓✓ Beats tuned baseline 2026

Differentiable R-Function Geometry Gate

Attach an analytic geometry gate to a KAN or MLP so that known feasible regions, exclusions, and unions are represented by differentiable implicit functions instead of being learned only from samples. Use R-conjunctions for intersections and R-disjunctions for unions, then convert the signed support score into a soft gate that modulates the prediction.

Useful7/10
Difficulty4/10
Novelty6/10
Paper: Geometry-Aware R-Structured Kolmogorov-Arnold Networks arXiv:2607.01449
✓✓ Beats tuned baseline 2026

Local Characteristic Residual Gating

Transform local neural residuals into the Ripa model's characteristic coordinates before spatial aggregation, apply a mode-dependent gate based on neighboring characteristic jumps, and transform back. This lets the model damp oscillatory acoustic or equilibrium-mode corrections near discontinuities without globally smoothing every feature.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Fifth-Order Well-Balanced Path-Conservative A-WENO Scheme for the Ripa Model arXiv:2607.09293
✓✓ Beats tuned baseline 2026

Finite-group relative message passing

Use the quotient group's generator classes as a finite relation vocabulary and tie message functions by group displacement instead of by individual graph edges. This creates a compact, exactly consistent relation-aware GNN that can recognize repeated local structure and transfer parameters across graph instances sharing the same Cayley geometry.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Minimal Isometric Embeddings of Graphs into Cayley Graphs of Finite Abelian Groups arXiv:2607.07920
✓✓ Beats tuned baseline 2026

Agnostic Geometry-Prior Mixer

Train an unconstrained branch and a geometry-aware branch in parallel, then learn how much to trust the analytic branch. This preserves the benefit of explicit geometry on correctly specified tasks while allowing the model to ignore a misleading or irrelevant prior.

Useful6/10
Difficulty3/10
Novelty7/10
Paper: Geometry-Aware R-Structured Kolmogorov-Arnold Networks arXiv:2607.01449
✓✓ Beats tuned baseline 2026

Jacobian-Ranked Simplex Features

Add a differentiable hypergraph layer that converts invariant edge-length features into triangle areas or higher-dimensional simplex volumes before message passing. Select or weight simplices according to the singular values of the length-to-volume Jacobian, so the network receives geometrically independent features rather than many redundant or nearly degenerate measurements.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: On volume vectors determined by hypergraphs in thin subsets of Euclidean space arXiv:2607.00153
✓✓ Beats tuned baseline 2026

Minkowski-Additive Convex Latents

Store a convex object as a direction-indexed vertex tuple and implement composition of objects through componentwise Minkowski addition and nonnegative scaling. This creates a structured residual or compositional layer where convexification is nonexpansive, making perturbation amplification controllable and avoiding repeated generic geometric optimization.

Useful6/10
Difficulty4/10
Novelty8/10
Paper: Galerkin approximations to the space of convex bodies by polytopes in nondegenerate V-representation arXiv:2608.26615
✓✓ Beats tuned baseline 2026

Continued-Fraction Lacunary Features

Replace random Fourier features or a dense sinusoidal positional encoding with a compact bank whose frequencies are the continued-fraction denominators of an irrational number. Inverse-frequency amplitudes provide multiscale structure with a controlled sub-Lipschitz regularity profile, while lacunarity reduces the number of frequencies needed to represent oscillatory structure.

Useful6/10
Difficulty3/10
Novelty5/10
Paper: Regularity, quantitative deviation, and non-rigidity of a lacunary skew product arXiv:2608.25821
✓✓ Beats tuned baseline 2026

Exact Elasticity-Complex Message Passing

Construct a mesh neural network with node, edge, face, and cell feature spaces modeled on the four spaces of the discrete elasticity complex. Replace unconstrained cross-order message passing by fixed incidence and geometric operators whose compositions vanish exactly, so gradient-like, incompatibility-like, and divergence-like features cannot contain algebraically spurious components.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: A Vector-Valued Co-Chain/Chain Complex Associated to the Elasticity Complex arXiv:2608.23829