Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

△ Mechanism confirmed, baseline not beaten 2026

Differentiable Asymmetric Admissibility Layer

Replace hard clipping or post-hoc asymmetric saturation with a dynamic output state that remains inside a prescribed asymmetric interval. A neural network emits a command uc, while the realized output u evolves through the APIR vector field, producing bounded actions, temporal smoothing, and gradients that remain available in the interior.

Useful8/10
Difficulty4/10
Novelty7/10
Paper: Admissibility-Preserving Control for Strict-Feedback Nonlinear Systems with Asymmetric Actuator Constraints arXiv:2608.15375
✓✓ Beats tuned baseline 2026

Nonlinearity-Subtracted Latent State-Space Model

Build a latent continuous-time neural model with dynamics \(\dot{z}=Az+f_\phi(z)\), where \(f_\phi\) is known, separately computed, or frozen, and \(A\) is learned exclusively from the derivative residual after subtracting \(f_\phi(z)\). Parameterize \(A\) with a truncated SVD or low-rank factorization so its eigenvalues directly predict local stability and long-horizon growth.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Data-driven linear analysis of dynamical systems via nonlinearity-subtracted dynamic mode decomposition arXiv:2608.13373
✗ Failed on benchmark 2026

Generator-Flow Equivariance Training

Use discovered infinitesimal generators to create small continuous transformations of hidden states and force a neural predictor to commute with those transformations. This converts symmetry discovery into self-supervised augmentation without prespecifying a group, canonical coordinates, or hand-designed equivariant layers.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: LieStoNet: Learning Lie Symmetries from Spatiotemporal Data for Stochastic Dynamical Systems arXiv:2608.01582
✗ Mechanism failed 2026

Matrix-Free Differentiable CBF Safety Layer

Attach a hard control-barrier-function quadratic-program safety filter to a neural policy, but solve the filter with operator splitting and differentiate through its fixed-point map using projection Jacobian-vector products. The network learns the nominal action and task objective end to end, while the deployed action remains the feasible filtered action rather than an unconstrained penalty-based approximation.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: End-to-End Learning of Safe Optimal Feedback Control in High Dimensions with Control Barrier Function Layers arXiv:2607.20674
✓✓ Beats tuned baseline 2026

Intrinsic Tangent-Projected Point-Cloud Layer

Modify a point-cloud message-passing or neural-operator layer so that scalar gradients, vector features, and vector-to-vector interactions are computed only in the estimated tangent plane of the surface. Projecting both feature values and derivative directions prevents the network from using arbitrary ambient-space normal directions and should improve transfer across differently embedded but intrinsically similar surfaces.

Useful7/10
Difficulty4/10
Novelty5/10
Paper: Solving the Incompressible Navier-Stokes Equations on Oriented Curved Surfaces Discretized by Point Clouds arXiv:2609.00216
✗ Failed on benchmark 2026

Rankine–Hugoniot Front Tokens

Augment a 1D neural operator or transformer with explicit tokens for detected discontinuities. Advance each front analytically using the local Rankine–Hugoniot speed and train the network only to reconstruct smooth regions and the residual caused by source terms and grid resolution.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Physics-informed token transformer methodology for nonlinear balance laws. I. Schwarzschild--Burgers fluid flows arXiv:2607.23143
△ Mechanism confirmed, baseline not beaten 2026

Nullspace-coordinate constrained operator blocks

Build a neural operator from frozen ambient mechanism blocks and a geometry-specific algebraic constraint adapter. The adapter parameterizes all outputs in the affine set satisfying sampled linear constraints exactly, so the network never produces boundary-violating states and does not require a penalty coefficient or post-step projection.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Geometry-aware LegONet for PDE Learning on Arbitrary Domains arXiv:2607.23069
✗ Failed on benchmark 2026

Fractional Memory State-Space Layer

Replace a standard recurrent state update or finite-order SSM filter with a causal relative-history operator using a weakly singular kernel k(s)=s^{p-1}m(s), where 0<p<1. The resulting layer retains information over a power-law range of timescales and introduces tunable frequency-dependent phase and attenuation, while remaining implementable through a small bank of exponentially decaying states.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: High frequency wave propagation for the viscoelastic wave equation with singular memory arXiv:2608.30138
✓✓ Beats tuned baseline 2026

Exact Moment Message Passing

Replace per-particle message evaluation in a point-cloud or particle-based neural layer with exact box moments. Particles inside a box are compressed into a fixed tensor of monomial sums, and every query in that box evaluates the same piecewise-polynomial interaction from those moments, reducing work from particle-query pairs to particles plus occupied boxes.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Exact hierarchical algorithms for accelerating particle--mesh coupling in sparse-grid particle-in-cell methods arXiv:2608.19702
△ Mechanism confirmed, baseline not beaten 2026

Quadrature-Whitened Neural Feature Subspace

Freeze a wide neural spatial dictionary, then compress and whiten it using the quadrature mass matrix before solving for output coefficients or latent PDE states. The retained basis removes feature directions that are numerically invisible or nearly dependent under the actual domain discretization, while preserving the represented function space up to the chosen SVD rank.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Evo-GTransNet for Parabolic PDEs: A Fixed-Feature Galerkin Method of Lines with Quadrature-Mass Orthonormalization arXiv:2608.19615
✗ Mechanism failed 2026

Exact-Curl Neural Field Output

Make a neural network predict a vector potential rather than a magnetic or velocity field, then obtain the physical vector field with a fixed differentiable discrete curl. The reconstructed field satisfies the discrete divergence-free constraint exactly, eliminating divergence-penalty tuning and preventing constraint drift during long rollouts.

Useful7/10
Difficulty4/10
Novelty5/10
Paper: A Structure- and Pressure-Positivity-Preserving Semi-implicit IMEX Finite Volume Scheme for Ideal MHD at All Acoustic Mach and Alfvén Mach Numbers with Generic Equation of State arXiv:2608.15837
✗ Mechanism failed 2026

FFT Weak-Residual Engine

Evaluate weak residuals against a bank of periodic trigonometric test functions using FFT projections instead of repeated pointwise quadrature or output automatic differentiation. Frequency truncation and mode weighting provide a direct way to control the spatial scales enforced during neural PDE training.

Useful7/10
Difficulty4/10
Novelty6/10
Paper: Efficient Weak-Entropy PINN for Solving Hyperbolic Conservation Laws arXiv:2608.10389
△ Mechanism confirmed, baseline not beaten 2026

Explicit Multi-Timescale Memory Bank

Use a bank of damped rotational state channels with a deliberately spread decay spectrum, allowing one recurrent layer to represent short, medium, and long temporal dependencies without relying on a single learned spectral radius. Concatenate the channels and train a readout or downstream nonlinear head to select the appropriate memory timescale.

Useful7/10
Difficulty4/10
Novelty7/10
Paper: Lindblad-Inspired Multi-Timescale Reservoir Computing with Separable Rotation and Dissipation arXiv:2608.04028
✓✓ Beats tuned baseline 2026

Parallel Quadratic Tree Layer

Replace sequential upward message passing in a tree-structured neural module with rake–compress contraction of quadratic latent-state messages. Each node stores a quadratic value function and each edge stores a linear transition or coupling triple; leaf elimination and unary-node compression are implemented as batched Schur complements, followed by a reverse pass that reconstructs node latents and edge outputs. The layer is exact for Gaussian or quadratic latent models and remains…

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Rake-Compress Riccati Recursions for Parallel Scenario-Tree Model Predictive Control arXiv:2608.01332
✗ Failed on benchmark 2026

Exact Event-Chained Neural ODE

Represent a hybrid trajectory with one neural module per known dynamical phase rather than a single network spanning all phases. Feed the predicted terminal state of phase r directly as the initial state of phase r+1, so continuity is satisfied by construction instead of by a soft interface penalty. This should improve learning near abrupt changes and remove an otherwise poorly conditioned loss-weight tradeoff.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Event-Structured Physics-Informed Neural Networks for Differentiable Critical Clearing Boundaries arXiv:2607.27681
△ Mechanism confirmed, baseline not beaten 2026

Invariant-domain learned reconstruction

Insert a neural local reconstruction into a finite-volume or graph-based simulator, but hard-cap its contribution so every reconstructed state remains in the physical admissible set. The network learns accuracy-sensitive gradients or stencil weights; a deterministic limiter, rather than a penalty loss, guarantees positive density and pressure for arbitrary network outputs.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Guarantees by Construction for Learned Finite Volume Schemes on Steady Supersonic Flow arXiv:2607.20171
△ Mechanism confirmed, baseline not beaten 2026

Monotone CDT autoencoder bottleneck

Build an autoencoder whose decoder outputs a monotone quantile function rather than an unconstrained spatial field. The latent representation can be compressed with POD or a neural bottleneck in CDT space, while the decoder guarantees valid transport maps and therefore avoids negative densities, mass drift, and spurious oscillations common in unconstrained reduced-order neural decoders.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Reduced Order Modeling of One-Dimensional Conservative PDEs via the Cumulative Distribution Transform arXiv:2607.17066
△ Mechanism confirmed, baseline not beaten 2026

Noise-Triggered Latent Rank Adaptation

Use the recursive errors-in-variables subspace spectrum as a controller for the width of a latent SSM rather than fixing the state dimension in advance. Neurons or state channels are added when corrected covariance eigenvalues rise above the noise floor and pruned when they remain below it, producing a model-order-adaptive recurrent architecture for nonstationary streams.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: A recursive subspace based method for errors-in-variables model identification of time-varying systems arXiv:2607.17065
△ Mechanism confirmed, baseline not beaten 2026

Constraint-preserving DAE neural block

Build a neural dynamical block whose hidden state contains differential variables and Lagrange multipliers, with a singular descriptor matrix enforcing constraints during propagation. This avoids the drift and ill-conditioning that can arise when exact constraints are represented only by a penalty term.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Contour integral methods and structured perturbations for linear differential-algebraic equations arXiv:2607.12628
✗ Mechanism failed 2026

Zeta-Corrected Singular Integral Layer

Construct a periodic neural integral layer whose fixed singular kernel behaves like |y|^{-s} near the origin, but whose samples on the uniform grid are replaced on a small symmetric stencil by SinCoTrap correction weights. The correction cancels low-order Taylor errors caused by sampling the singularity, while all nonlocal grid points remain unchanged. Increasing the correction order from p=0 to p=1 or p=2 should reduce discretization error without increasing global grid resolution.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: SinCoTrap: A High-Order Locally Corrected Trapezoidal Rule for Periodic Singular Integrals in Arbitrary Dimensions arXiv:2607.12390
✓✓ Beats tuned baseline 2026

Multiplicative Manifold Unscented Recurrent Cell

Replace the Euclidean hidden-state update of a recurrent or state-space neural network with a mixed manifold state containing a rotation component and Euclidean features. Propagate uncertainty with sigma points in tangent error coordinates, retract rotational perturbations through the exponential map, and compute the training loss from the predicted covariance. This avoids invalid rotations and captures second-order curvature effects that a first-order EKF-style recurrent cell misses at large…

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Fully Multiplicative Attitude and Orbit Determination for Deep space Navigation arXiv:2607.10072
✗ Failed on benchmark 2026

Equilibrium-Coordinate Neural Operator

Build a neural PDE surrogate that predicts changes in equilibrium variables rather than changes in conservative state variables. The network receives the local state and geometry, predicts an equilibrium-coordinate increment, and subtracts the network output evaluated at a reference equilibrium, forcing the reference state to have exactly zero learned residual.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Fifth-Order Well-Balanced Path-Conservative A-WENO Scheme for the Ripa Model arXiv:2607.09293
✗ Mechanism failed 2026

DMDc-Initialized Marginal-Stable Neural SSM

Replace the unconstrained transition of a recurrent or state-space neural network with a DMDc-initialized linear latent transition plus a learned nonlinear residual. Estimate the transition from a short warm-up dataset using Hankel delay coordinates, retain eigenmodes with decay rates near the unit circle for long-term memory, and let the neural residual model dynamics not explained by the linear backbone. This should make long-horizon prediction and slowly varying signals easier to learn while…

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Koopman Spectral Analysis of Lithium-Ion Battery Dynamics: State of Charge as a Marginally Stable Observable arXiv:2607.07594
✗ Failed on benchmark 2026

Conservative Sparse Mortar Cross-Attention

Replace dense coarse-to-fine cross-attention at multiresolution interfaces with a sparse, nonnegative overlap operator whose weighted feature average is exactly conserved between the two resolutions. Use this operator as a low-order path and blend it with an unrestricted neural cross-attention path through a convex limiter that keeps features inside a box or simplex domain. The construction is especially suitable for adaptive token grids, hierarchical graph neural networks, neural operators…

Useful7/10
Difficulty5/10
Novelty8/10
Paper: Invariant-domain-preserving limiting with Adaptive Mesh Refinement for Legendre-Gauss-Lobatto Discontinuous Galerkin Spectral Element Methods arXiv:2607.06045