Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✗ Failed on benchmark 2026

Generator-Flow Equivariance Training

Use discovered infinitesimal generators to create small continuous transformations of hidden states and force a neural predictor to commute with those transformations. This converts symmetry discovery into self-supervised augmentation without prespecifying a group, canonical coordinates, or hand-designed equivariant layers.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: LieStoNet: Learning Lie Symmetries from Spatiotemporal Data for Stochastic Dynamical Systems arXiv:2608.01582
✗ Failed on benchmark 2026

Rankine–Hugoniot Front Tokens

Augment a 1D neural operator or transformer with explicit tokens for detected discontinuities. Advance each front analytically using the local Rankine–Hugoniot speed and train the network only to reconstruct smooth regions and the residual caused by source terms and grid resolution.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Physics-informed token transformer methodology for nonlinear balance laws. I. Schwarzschild--Burgers fluid flows arXiv:2607.23143
✗ Failed on benchmark 2026

Fractional Memory State-Space Layer

Replace a standard recurrent state update or finite-order SSM filter with a causal relative-history operator using a weakly singular kernel k(s)=s^{p-1}m(s), where 0<p<1. The resulting layer retains information over a power-law range of timescales and introduces tunable frequency-dependent phase and attenuation, while remaining implementable through a small bank of exponentially decaying states.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: High frequency wave propagation for the viscoelastic wave equation with singular memory arXiv:2608.30138
✗ Failed on benchmark 2026

Shared-response expert ranking

Build a label-free router for a finite library of neural operators by estimating one shared physical target response from an anchor prediction and using it to rank every candidate through inner products with candidate differences. The method avoids running a full residual-based diagnostic independently for every expert and can be used either to select the best expert or to form a corrected weighted combination.

Useful7/10
Difficulty6/10
Novelty8/10
Paper: Shared Physics Responses Recover Hidden Rankings in Neural Operator Libraries arXiv:2608.20441
✗ Failed on benchmark 2026

Exact Event-Chained Neural ODE

Represent a hybrid trajectory with one neural module per known dynamical phase rather than a single network spanning all phases. Feed the predicted terminal state of phase r directly as the initial state of phase r+1, so continuity is satisfied by construction instead of by a soft interface penalty. This should improve learning near abrupt changes and remove an otherwise poorly conditioned loss-weight tradeoff.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Event-Structured Physics-Informed Neural Networks for Differentiable Critical Clearing Boundaries arXiv:2607.27681
✗ Failed on benchmark 2026

Spectral-Band Dual-Timescale Network

Split hidden dynamics into relaxation bands when the Jacobian spectrum has a gap, evolve each band with its own timescale, and retain an explicit cross-band exchange term. This yields a principled dual-timescale RNN or SSM rather than choosing fast and slow branches heuristically.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Richards' equation as a hydrodynamic limit: Chapman--Enskog reduction of the continuum kinetic equation for unsaturated soil water arXiv:2607.17358
✗ Failed on benchmark 2026

ISS-Constrained Modular Recurrent Network

Replace an unconstrained recurrent block with two coupled modules: a contractive perceptual estimator and an input-to-state-stable cognitive state transition. Spectral normalization and a controlled Euler residual step enforce a quantitative gain condition, preventing hidden-state explosion while retaining long memory when the contraction factor is chosen close to one.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: A modular state-space model of human perception, cognition, and decision dynamics arXiv:2607.14078
✗ Failed on benchmark 2026

Hard-Constrained Bernstein Memory Head

Add a causal memory branch whose lag-response function is represented by a Bernstein polynomial with coefficients constrained to produce a nonnegative, decreasing, convex kernel. The branch aggregates past hidden states using this kernel, giving the model a learnable long-memory profile while preventing oscillatory, negative, or increasing historical influence.

Useful7/10
Difficulty4/10
Novelty7/10
Paper: Neural Discovery of Memory and Nonlocal Kernels in Integro-Differential Equations with Constrained Kolmogorov--Arnold Networks arXiv:2607.11110
✗ Failed on benchmark 2026

Equilibrium-Coordinate Neural Operator

Build a neural PDE surrogate that predicts changes in equilibrium variables rather than changes in conservative state variables. The network receives the local state and geometry, predicts an equilibrium-coordinate increment, and subtracts the network output evaluated at a reference equilibrium, forcing the reference state to have exactly zero learned residual.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Fifth-Order Well-Balanced Path-Conservative A-WENO Scheme for the Ripa Model arXiv:2607.09293
✗ Failed on benchmark 2026

FSAL Runge-Kutta Neural Block

Replace a weight-tied residual or neural-ODE stepper with an explicit Runge–Kutta method satisfying the reused-last-stage conditions. The final derivative is evaluated at the exact endpoint and becomes the first derivative of the next step, saving one expensive neural-vector-field call per step while preserving the designed integration order.

Useful7/10
Difficulty5/10
Novelty5/10
Paper: On the order of Runge Kutta methods reusing last stage arXiv:2607.06788
✗ Failed on benchmark 2026

Conservative Sparse Mortar Cross-Attention

Replace dense coarse-to-fine cross-attention at multiresolution interfaces with a sparse, nonnegative overlap operator whose weighted feature average is exactly conserved between the two resolutions. Use this operator as a low-order path and blend it with an unrestricted neural cross-attention path through a convex limiter that keeps features inside a box or simplex domain. The construction is especially suitable for adaptive token grids, hierarchical graph neural networks, neural operators…

Useful7/10
Difficulty5/10
Novelty8/10
Paper: Invariant-domain-preserving limiting with Adaptive Mesh Refinement for Legendre-Gauss-Lobatto Discontinuous Galerkin Spectral Element Methods arXiv:2607.06045
✗ Failed on benchmark 2026

GQL Safe Residual Layer

Insert a scalar flux-correction-style limiter after a neural operator predicts a conservative state or residual. Interpolate between a known-admissible baseline state and the learned high-order candidate, choosing the largest coefficient that satisfies a geometric family of linear inequalities encoding positive density, positive pressure, and subluminal velocity. This retains as much of the neural prediction as possible instead of independently clipping physical variables.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: GQL-Based Physical-Constraint-Preserving High-Order Finite Difference Schemes for Special Relativistic Hydrodynamics in Arbitrary Dimensions arXiv:2606.31992