Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✓✓ Beats tuned baseline 2026

Conservative Flux Neural Operator

Replace pointwise prediction of the next field with prediction of a learned flux followed by a discrete divergence. Combine Fourier spatial mixing with a causal temporal kernel over the recent resolved-history slab, so the model learns finite-memory closure effects while preserving local conservation exactly under periodic or compatible boundary conditions. The architecture should reduce spurious mass drift and improve autoregressive rollout stability on coarse-grained PDE data.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: Flux-form spatiotemporal neural operators for coarse-grained dynamics of multiscale PDEs arXiv:2608.18148
△ Mechanism confirmed, baseline not beaten 2026

Differentiable Asymmetric Admissibility Layer

Replace hard clipping or post-hoc asymmetric saturation with a dynamic output state that remains inside a prescribed asymmetric interval. A neural network emits a command uc, while the realized output u evolves through the APIR vector field, producing bounded actions, temporal smoothing, and gradients that remain available in the interior.

Useful8/10
Difficulty4/10
Novelty7/10
Paper: Admissibility-Preserving Control for Strict-Feedback Nonlinear Systems with Asymmetric Actuator Constraints arXiv:2608.15375
△ Mechanism confirmed, baseline not beaten 2026

MCIS Safety Shield for Neural Controllers

Compute an inner approximation of the states from which a neural controller can keep the plant inside a prescribed safe domain indefinitely, then use the resulting regulation map as a safety shield around the network. At each state, the network proposes an action, but the shield projects or replaces it with an action certified to remain in the invariant set.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: Computing the Maximal Controlled Invariant Set for Neural Network Control Systems arXiv:2608.07908
✗ Failed on benchmark 2026

Generator-Flow Equivariance Training

Use discovered infinitesimal generators to create small continuous transformations of hidden states and force a neural predictor to commute with those transformations. This converts symmetry discovery into self-supervised augmentation without prespecifying a group, canonical coordinates, or hand-designed equivariant layers.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: LieStoNet: Learning Lie Symmetries from Spatiotemporal Data for Stochastic Dynamical Systems arXiv:2608.01582
✓✓ Beats tuned baseline 2026

Intrinsic Tangent-Projected Point-Cloud Layer

Modify a point-cloud message-passing or neural-operator layer so that scalar gradients, vector features, and vector-to-vector interactions are computed only in the estimated tangent plane of the surface. Projecting both feature values and derivative directions prevents the network from using arbitrary ambient-space normal directions and should improve transfer across differently embedded but intrinsically similar surfaces.

Useful7/10
Difficulty4/10
Novelty5/10
Paper: Solving the Incompressible Navier-Stokes Equations on Oriented Curved Surfaces Discretized by Point Clouds arXiv:2609.00216
✗ Mechanism failed 2026

Exact-Curl Neural Field Output

Make a neural network predict a vector potential rather than a magnetic or velocity field, then obtain the physical vector field with a fixed differentiable discrete curl. The reconstructed field satisfies the discrete divergence-free constraint exactly, eliminating divergence-penalty tuning and preventing constraint drift during long rollouts.

Useful7/10
Difficulty4/10
Novelty5/10
Paper: A Structure- and Pressure-Positivity-Preserving Semi-implicit IMEX Finite Volume Scheme for Ideal MHD at All Acoustic Mach and Alfvén Mach Numbers with Generic Equation of State arXiv:2608.15837
△ Mechanism confirmed, baseline not beaten 2026

Bundle-Glued Neural Field

Represent a field on a manifold with one neural network per chart, while enforcing the exact transition law between chart outputs on overlaps. This avoids the artificial requirement that one coordinate frame work globally and should improve learning on spherical, periodic, or otherwise topologically nontrivial domains. Use an augmented Lagrangian rather than only a pointwise penalty so chart compatibility is enforced strongly without requiring identical local parameterizations.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Finite element discretization of Yang--Mills connections arXiv:2608.02108
△ Mechanism confirmed, baseline not beaten 2026

Invariant-domain learned reconstruction

Insert a neural local reconstruction into a finite-volume or graph-based simulator, but hard-cap its contribution so every reconstructed state remains in the physical admissible set. The network learns accuracy-sensitive gradients or stencil weights; a deterministic limiter, rather than a penalty loss, guarantees positive density and pressure for arbitrary network outputs.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Guarantees by Construction for Learned Finite Volume Schemes on Steady Supersonic Flow arXiv:2607.20171
✗ Failed on benchmark 2026

ISS-Constrained Modular Recurrent Network

Replace an unconstrained recurrent block with two coupled modules: a contractive perceptual estimator and an input-to-state-stable cognitive state transition. Spectral normalization and a controlled Euler residual step enforce a quantitative gain condition, preventing hidden-state explosion while retaining long memory when the contraction factor is chosen close to one.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: A modular state-space model of human perception, cognition, and decision dynamics arXiv:2607.14078
△ Mechanism confirmed, baseline not beaten 2026

Constraint-preserving DAE neural block

Build a neural dynamical block whose hidden state contains differential variables and Lagrange multipliers, with a singular descriptor matrix enforcing constraints during propagation. This avoids the drift and ill-conditioning that can arise when exact constraints are represented only by a penalty term.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Contour integral methods and structured perturbations for linear differential-algebraic equations arXiv:2607.12628
✗ Failed on benchmark 2026

Hard-Constrained Bernstein Memory Head

Add a causal memory branch whose lag-response function is represented by a Bernstein polynomial with coefficients constrained to produce a nonnegative, decreasing, convex kernel. The branch aggregates past hidden states using this kernel, giving the model a learnable long-memory profile while preventing oscillatory, negative, or increasing historical influence.

Useful7/10
Difficulty4/10
Novelty7/10
Paper: Neural Discovery of Memory and Nonlocal Kernels in Integro-Differential Equations with Constrained Kolmogorov--Arnold Networks arXiv:2607.11110
✗ Failed on benchmark 2026

Equilibrium-Coordinate Neural Operator

Build a neural PDE surrogate that predicts changes in equilibrium variables rather than changes in conservative state variables. The network receives the local state and geometry, predicts an equilibrium-coordinate increment, and subtracts the network output evaluated at a reference equilibrium, forcing the reference state to have exactly zero learned residual.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Fifth-Order Well-Balanced Path-Conservative A-WENO Scheme for the Ripa Model arXiv:2607.09293
△ Mechanism confirmed, baseline not beaten 2026

Dirichlet Spectral Projection Layers

Replace soft boundary penalties in neural operators with a hard projection onto a finite-dimensional span of homogeneous Dirichlet Laplacian eigenfunctions. Every projected hidden field is identically zero on the boundary, while increasing the number of retained eigenfunctions recovers the expressive capacity needed for operator approximation.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Enforcing Dirichlet Boundary Conditions in Operator Learning arXiv:2608.27256
Unverified 2026

Sobolev-Orthogonal MLP Features

Replace raw polynomial or Fourier-like features in a small MLP with basis functions orthonormal under a Sobolev inner product that jointly measures feature magnitude and input derivative magnitude. This explicitly controls feature smoothness while preserving decorrelation, potentially improving conditioning and reducing the need for large derivative-regularization coefficients.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: A Riemann-Hilbert representation for Sobolev orthogonal polynomials arXiv:2608.08397
Unverified 2026

Log-Fractional Scale-Derivative Layer

Add a feature transformation that approximates the derivative of a fractional diffusion operator with respect to its order. Instead of only smoothing features with one fractional order, the layer exposes whether a feature changes rapidly across spatial scales, which can help with textures, edges, and multiscale patterns.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Regularity for the fractional logarithmic $p$-Laplacian arXiv:2607.11462
✓✓ Beats tuned baseline 2026

Local Characteristic Residual Gating

Transform local neural residuals into the Ripa model's characteristic coordinates before spatial aggregation, apply a mode-dependent gate based on neighboring characteristic jumps, and transform back. This lets the model damp oscillatory acoustic or equilibrium-mode corrections near discontinuities without globally smoothing every feature.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Fifth-Order Well-Balanced Path-Conservative A-WENO Scheme for the Ripa Model arXiv:2607.09293
✗ Mechanism failed 2026

Fourier-Calibrated Nonlocal Feature Gradient

Augment a CNN with a nonlocal feature-gradient branch that compares each feature vector with a kernel-weighted neighborhood rather than using only pointwise or local convolutional interactions. Regularize this branch using the paper's Fourier multiplier energy, which penalizes feature oscillations according to the kernel spectrum and approaches an ordinary local-gradient operator as the interaction radius tends to zero.

Useful6/10
Difficulty4/10
Novelty6/10
Paper: A Variational Nonlocal Phase-Field Model for Dynamic Fracture in Elastic Solids arXiv:2607.01881
✗ Failed on benchmark 2026

GQL Safe Residual Layer

Insert a scalar flux-correction-style limiter after a neural operator predicts a conservative state or residual. Interpolate between a known-admissible baseline state and the learned high-order candidate, choosing the largest coefficient that satisfies a geometric family of linear inequalities encoding positive density, positive pressure, and subluminal velocity. This retains as much of the neural prediction as possible instead of independently clipping physical variables.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: GQL-Based Physical-Constraint-Preserving High-Order Finite Difference Schemes for Special Relativistic Hydrodynamics in Arbitrary Dimensions arXiv:2606.31992
Unverified 2026

Convex-order stochastic expert layer

Replace a deterministic mixture-of-experts residual block with K population-indexed stochastic expert states coupled through a graphon matrix. The layer uses a shared drift and expert-dependent diffusion, while an empirical convex-order penalty makes later representations more dispersed than a reference representation without permitting a mean shift.

Useful5/10
Difficulty5/10
Novelty7/10
Paper: Convex order preservation for graphon mean-field systems arXiv:2608.19576
Unverified 2026

Mean-Curvature Relaxation Layer

Insert a small number of differentiable graphical mean-curvature-flow steps between a neural network's raw vector-field prediction and its task loss. The relaxation performs geometry-aware smoothing rather than isotropic Gaussian smoothing, and it can enforce fixed boundary values after every step.

Useful5/10
Difficulty5/10
Novelty7/10
Paper: Well-posedness for the mean curvature flow on the half-space and on bounded domains arXiv:2608.08901
Unverified 2026

Metric-magnitude pooling

Replace mean or max pooling over a set of learned element embeddings with pooling based on the metric-magnitude weighting. Pairwise distances create a globally coupled correction for redundancy, so geometrically isolated or boundary elements can contribute differently from dense clusters of nearly duplicate elements.

Useful5/10
Difficulty5/10
Novelty8/10
Paper: Scalably computing metric magnitude arXiv:2607.23354
Unverified 2026

LU-Preconditioned Orthogonal Weight Retraction

Periodically project a rectangular neural-network weight matrix onto an approximately orthonormal-column matrix using LU-preconditioned CholeskyQR rather than ordinary QR or a polar iteration. Pivoted LU handles badly scaled and nearly dependent columns, while Householder orthogonalization of the LU factor produces a triangular preconditioner that makes the subsequent Cholesky step safer in fp16 or bfloat16.

Useful5/10
Difficulty6/10
Novelty5/10
Paper: RCLUPPr: a new randomized CholeskyQR with LU preconditioning arXiv:2607.15561