Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

△ Mechanism confirmed, baseline not beaten 2026

Farkas-Certified Neural Safety Shield

Insert a constraint-reduction layer between a policy network and its executed action. The policy proposes an action, while the layer retains only geometrically extreme collision and obstacle constraints and verifies that every discarded halfspace is implied by the retained ones through nonnegative conic multipliers. The reduced projection or quadratic program is therefore equivalent to the full tightened safety filter whenever certification succeeds, but uses substantially fewer constraints.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Scalable Tube-Tightened Multi-Agent Safety via Certified Constraint Reduction arXiv:2608.25323
△ Mechanism confirmed, baseline not beaten 2026

MCIS Safety Shield for Neural Controllers

Compute an inner approximation of the states from which a neural controller can keep the plant inside a prescribed safe domain indefinitely, then use the resulting regulation map as a safety shield around the network. At each state, the network proposes an action, but the shield projects or replaces it with an action certified to remain in the invariant set.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: Computing the Maximal Controlled Invariant Set for Neural Network Control Systems arXiv:2608.07908
△ Mechanism confirmed, baseline not beaten 2026

Positive Curvature Mirror-Descent DEQ

Replace the black-box equilibrium denoiser in an image-restoration DEQ with a positivity-preserving mirror-descent equilibrium driven by the exact Gamma likelihood and a discretized surface-area/mean-curvature regularizer. The equilibrium layer has a small number of learned scalar or channel-wise parameters instead of a large implicit CNN, while the exponentiated update prevents negative intensities and naturally matches multiplicative noise.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: A geometry-based deep equilibrium model for image restoration under multiplicative Gamma noise arXiv:2608.04944
△ Mechanism confirmed, baseline not beaten 2026

Legendre Feasibility Layer

Replace an unconstrained neural decision head followed by clipping, projection, or penalty losses with a Legendre-regularized optimizer layer that always returns a feasible decision. The layer maps network-produced latent scores smoothly into the relative interior of a convex feasible region and can be differentiated using the inverse Hessian of the regularizer.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: Smooth Learning with Hard Constraints via Legendre-Regularized Policies arXiv:2607.24007
△ Mechanism confirmed, baseline not beaten 2026

Monotone transport-map latent space

Represent every nonnegative equal-mass one-dimensional state by its CDT quantile map relative to a fixed reference density, then train the neural dynamics model in this transformed space rather than on Eulerian grid values. The latent manifold for translations and transport-dominated evolution is substantially flatter: linear transport lies in the span of the initial transformed state and the constant function, while nonlinear conservative dynamics have algebraic approximation error bounds.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: Reduced Order Modeling of One-Dimensional Conservative PDEs via the Cumulative Distribution Transform arXiv:2607.17066
△ Mechanism confirmed, baseline not beaten 2026

Biclique-Hub Attention

Replace a dense directed attention matrix by a collection of K learned source-to-hub-to-target interactions. Each hub corresponds to a directed biclique, allowing many source tokens to communicate with many target tokens using O(NK) rather than O(N^2) pair interactions. The construction preserves asymmetric information flow and can be initialized from a graph cover of high-attention edges.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: On Transformer Dynamics arXiv:2607.13295
△ Mechanism confirmed, baseline not beaten 2026

Pole-residue latent diffusion decoder

Represent a one-dimensional sharp-feature signal by a small unordered set of complex singularities and residues instead of predicting all grid amplitudes. A transformer diffusion model predicts these tokens, and a differentiable meromorphic decoder evaluates the result directly at arbitrary coordinates, avoiding grid-specific interpolation and preserving discontinuity structure.

Useful8/10
Difficulty6/10
Novelty8/10
Paper: The Singularity Space: A Generative Diffusion Framework for Signal Representation arXiv:2607.10930
△ Mechanism confirmed, baseline not beaten 2026

Quotient Spectral Positional Encoding

Construct a graph and its spectral positional features using affinities between inputs after optimally aligning one input over the known symmetry group. Feed these quotient-space eigenvectors to a transformer or graph neural network, so symmetry-equivalent examples receive the same structural coordinates without storing augmented copies.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: Group Invariant Spectral Embedding arXiv:2607.08987
△ Mechanism confirmed, baseline not beaten 2026

Parity-block curvature preconditioner

When a model, dataset, or parameterization has an involutive symmetry, transform gradients and curvature into symmetry sectors before applying adaptive updates. The Hessian and optimizer can then be handled as independent even and odd blocks, preventing curvature from one sector from forcing an unnecessarily conservative learning rate in the other.

Useful7/10
Difficulty4/10
Novelty6/10
Paper: The Hessian of Planar Central Configurations in Pair Space: Decomposition, Morse Index and Symmetry Reduction arXiv:2609.01857
△ Mechanism confirmed, baseline not beaten 2026

Piola-Conditioned Fixed-Reference Neural Operator

Build a geometry-conditioned neural operator on a single reference mesh instead of remeshing or changing the network discretization for every domain shape. Transport vector-valued surface features with a contravariant surface Piola map, and feed the network geometry-dependent pulled-back quantities. This should make the architecture stable across shape changes and allow batching many geometries with identical tensor shapes.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Shape Holomorphy and Sparse Approximation of the Maxwell Electric Field Integral Operator arXiv:2609.00466
△ Mechanism confirmed, baseline not beaten 2026

Nullspace-coordinate constrained operator blocks

Build a neural operator from frozen ambient mechanism blocks and a geometry-specific algebraic constraint adapter. The adapter parameterizes all outputs in the affine set satisfying sampled linear constraints exactly, so the network never produces boundary-violating states and does not require a penalty coefficient or post-step projection.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Geometry-aware LegONet for PDE Learning on Arbitrary Domains arXiv:2607.23069
△ Mechanism confirmed, baseline not beaten 2026

Hypoelliptic transport-diffusion layer

Replace an isotropic local mixing layer with a kinetic layer that smooths features in x and transports them in y along the characteristic direction x. The layer should be useful for phase-space data, learned simulators, and world models in which positions or transported quantities evolve through coupled drift and diffusion rather than independent Euclidean motion.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Boundary Harnack inequalities for Kolmogorov equations in asymptotically cylindrical Lipschitz domains arXiv:2608.29813
△ Mechanism confirmed, baseline not beaten 2026

Harmonic-Mode Branch for Topological Memory

Do not force Hodge dissipation onto harmonic edge modes, because these modes are precisely the obstruction to global coercivity. Split the latent state into dissipative coexact modes and a finite-dimensional harmonic branch, and use harmonic-decoupled interactions so each harmonic coordinate defines an invariant affine fibre with its own attractor.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Hodge Coercivity and Global Dynamics in Two-Field Edge-Cochain Systems with MHD-Type Cancellation arXiv:2608.19360
△ Mechanism confirmed, baseline not beaten 2026

Ordered Diffusion Message Passing

Use a learned scalar ordering function to turn a symmetric local Gaussian graph kernel into a directed, row-stochastic message-passing operator. The asymmetric tilt lets neighboring nodes communicate preferentially along an inferred dynamical direction, while the Gaussian factor retains locality and diffusion-like smoothing.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Ordered Diffusion Kernels arXiv:2608.18019
△ Mechanism confirmed, baseline not beaten 2026

Dimension-Free Brenier Transport Layer

Build a neural transport layer by parameterizing a convex potential whose gradient maps a semi-log-concave latent distribution into a compact convex data domain. Use the paper's dimension-free Lipschitz certificate to set the layer's Jacobian scale, initialize the potential, and reject or regularize parameter updates that create excessive curvature. The goal is a bounded-output transport module that is less sensitive to latent dimension than diameter-based spectral heuristics.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Dimension-Free Lipschitz Bounds for Brenier Maps to Compactly Supported Log-Concave Targets arXiv:2608.15906
△ Mechanism confirmed, baseline not beaten 2026

Gram-Whitened Directional Pooling

Represent local feature channels as a smooth directional signal and aggregate them with a partition-of-unity family of learnable spherical atoms instead of hard angular bins. Use the atom Gram matrix to whiten the descriptor and add a projected-energy loss, so the network is rewarded for retaining information in the directional subspace rather than merely producing large correlated channel responses.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Point Feature Descriptor via Directional Partition of Unity on Maps arXiv:2608.12794
△ Mechanism confirmed, baseline not beaten 2026

Measurement-Space Neural Operator with Mesh Transfer

Build a neural operator around explicit input and output measurement spaces rather than forcing the network to consume and emit a fixed grid. The same learned latent surrogate can be reused on alternative sensor layouts or query meshes through reconstruction and re-encoding maps, with a consistency loss enforcing agreement between measurement pipelines.

Useful7/10
Difficulty6/10
Novelty5/10
Paper: Kernel Methods for Learning Operators with Multiple Inputs and Outputs arXiv:2608.11831
△ Mechanism confirmed, baseline not beaten 2026

Graded Levy-area recurrent state

Replace additive recurrent pooling with a graded state containing the current feature increment, an antisymmetric order-sensitive area matrix, and an optional symmetric quadratic-variation accumulator. Compose chunks using the paper's exact group law, allowing a sequence model to retain compressed pairwise ordering information without explicitly forming all token pairs.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Infinite-Dimensional Levy Area: Probability-Selected Critical Geometry and Sharp Spectral Selection arXiv:2608.08756
△ Mechanism confirmed, baseline not beaten 2026

Finite Hyperplane Representative Verification

Replace dense continuous action search during neural-controller verification with a finite set of representative inputs induced by affine pieces of the interval neural dynamics. This makes safety checking parallel over state cells and candidate actions, enabling much cheaper certification or repeated safe-set updates.

Useful7/10
Difficulty7/10
Novelty8/10
Paper: Computing the Maximal Controlled Invariant Set for Neural Network Control Systems arXiv:2608.07908
△ Mechanism confirmed, baseline not beaten 2026

Toda-Krylov adaptive polynomial layer

Replace a fixed-order polynomial or recurrent state-space block by an Arnoldi basis built from a learned operator and the current input, and use subdiagonal coefficients as geometry-aware gates over Krylov depth. The gates quantify how much genuinely new direction each operator application contributes, allowing the layer to stop early near Krylov breakdown and suppress redundant or unstable directions.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Two-dimensional Toda--Arnoldi correspondence: Holomorphic Krylov geometry and counterdiabatic transport arXiv:2608.04850
△ Mechanism confirmed, baseline not beaten 2026

Isometric tensor-network token mixer

Use the relaxed QFT tensor-network topology as a trainable norm-preserving mixer inside a neural block, replacing a dense token-mixing matrix or an expensive global convolution. The network learns data-adapted global interactions while retaining structured O(N log^2 N) application and an exact cheap inverse, making it suitable for image tokens, long sequences, or reversible residual blocks.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Fast Trainable Multilinear Bases for Image Compression arXiv:2608.00053
△ Mechanism confirmed, baseline not beaten 2026

Exact-boundary geometric PINN ansatz

Replace penalty enforcement of boundary and asymptotic conditions by a neural-network parametrization that satisfies them identically. For a disc mapped to hyperbolic half-space, construct the positive height coordinate with a vanishing boundary factor and construct tangential coordinates as a fixed boundary extension plus a vanishing correction.

Useful7/10
Difficulty4/10
Novelty6/10
Paper: A user's guide to PINNs in geometric analysis: lessons from the asymptotic Plateau problem arXiv:2607.28733
△ Mechanism confirmed, baseline not beaten 2026

Feasible Action Mapping Safety Layer

Let a neural policy emit an unconstrained abstract action z, then solve a state-dependent feasibility problem that maps z to an admissible optimal-control parameter p before execution. Unlike coordinate-wise clipping, the mapping accounts for predicted dynamics, coupled state and input constraints, and recursive feasibility, allowing the policy to retain a simple unconstrained output space while the controller enforces plant constraints.

Useful7/10
Difficulty6/10
Novelty5/10
Paper: Bridging Reinforcement Learning and Optimal Control via Feasible Action Mapping arXiv:2607.23930
△ Mechanism confirmed, baseline not beaten 2026

Tangent-Branch Neural Evasion Layer

Wrap a neural multi-agent policy with an analytic planner that generates turn-straight trajectories tangent to pursuer surveillance disks, then selects the branch with the smallest predicted completion time. The network supplies high-level preferences or residual corrections, while the geometric layer prevents unnecessarily entering exclusion regions and exposes an explicit branch-switching signal for training.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Semi-Explicit Solutions to the Prying-Pedestrian Surveillance-Evasion Differential Game and Extensions to Two Pursuers arXiv:2607.21087