Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✗ Failed on benchmark 2026

Non-Abelian Event-Order Memory

Augment an RNN or state-space model with a three-dimensional auxiliary spin updated by noncommuting rotations associated with event types or token classes. The ordered product preserves information that additive counters discard: two sequences with the same number of each event can produce different final spins when their event order differs. Train the spin axes, angles, and readout jointly with the task model while constraining every update to remain on the sphere.

Useful8/10
Difficulty5/10
Novelty8/10
Paper: Non-Abelian Spin Counting of Ordered Stochastic Trajectories: Reentrant Finite-Time Chern Numbers arXiv:2608.23533
✗ Failed on benchmark 2026

Cubic-Group Averaged 3D Convolution

Constrain the first convolutional layer, or every convolutional layer, by averaging each kernel over the 48 rotations and reflections of the cubic point group. A scalar 3D field then receives exactly the same prediction after any lattice rotation or reflection, eliminating the need to learn equivalent crystallographic orientations from separate examples.

Useful8/10
Difficulty4/10
Novelty5/10
Paper: Cubic-Equivariant Neural Density Functional Theory for Three-Dimensional Lattice Fluids arXiv:2608.08137
△ Mechanism confirmed, baseline not beaten 2026

Monotone transport-map latent space

Represent every nonnegative equal-mass one-dimensional state by its CDT quantile map relative to a fixed reference density, then train the neural dynamics model in this transformed space rather than on Eulerian grid values. The latent manifold for translations and transport-dominated evolution is substantially flatter: linear transport lies in the span of the initial transformed state and the constant function, while nonlinear conservative dynamics have algebraic approximation error bounds.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: Reduced Order Modeling of One-Dimensional Conservative PDEs via the Cumulative Distribution Transform arXiv:2607.17066
△ Mechanism confirmed, baseline not beaten 2026

Quotient Spectral Positional Encoding

Construct a graph and its spectral positional features using affinities between inputs after optimally aligning one input over the known symmetry group. Feed these quotient-space eigenvectors to a transformer or graph neural network, so symmetry-equivalent examples receive the same structural coordinates without storing augmented copies.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: Group Invariant Spectral Embedding arXiv:2607.08987
✗ Mechanism failed 2026

Adversarial Subspace Residual Localizer

Replace a fragile learned similarity score for continuous object coordinates with a projection residual against a learned signal subspace. Candidate coordinates are represented by normalized Fourier or positional feature vectors, and the score is the fraction of feature energy outside the estimated subspace. The score remains useful even when the estimated subspace is adversarially rotated, because the perturbation is controlled directly by a sine-theta distance rather than by assumptions about…

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Certified Spherical MUSIC for 3D Localization under Adversarial Subspace Perturbations arXiv:2609.03264
△ Mechanism confirmed, baseline not beaten 2026

Nullspace-coordinate constrained operator blocks

Build a neural operator from frozen ambient mechanism blocks and a geometry-specific algebraic constraint adapter. The adapter parameterizes all outputs in the affine set satisfying sampled linear constraints exactly, so the network never produces boundary-violating states and does not require a penalty coefficient or post-step projection.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Geometry-aware LegONet for PDE Learning on Arbitrary Domains arXiv:2607.23069
✗ Mechanism failed 2026

Orbit-Consistent Equivariant Distillation

Constrain a student policy to transform its action in the same way that the input state is transformed, while constraining its value estimate to remain unchanged. During distillation, augment every teacher-student pair with several symmetry-transformed copies and penalize disagreement after transforming the student action back to the original frame.

Useful7/10
Difficulty5/10
Novelty4/10
Paper: SymVD: Symmetric Vision Language Action Distillation for Robot Manipulation arXiv:2608.29828
△ Mechanism confirmed, baseline not beaten 2026

Hypoelliptic transport-diffusion layer

Replace an isotropic local mixing layer with a kinetic layer that smooths features in x and transports them in y along the characteristic direction x. The layer should be useful for phase-space data, learned simulators, and world models in which positions or transported quantities evolve through coupled drift and diffusion rather than independent Euclidean motion.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Boundary Harnack inequalities for Kolmogorov equations in asymptotically cylindrical Lipschitz domains arXiv:2608.29813
△ Mechanism confirmed, baseline not beaten 2026

Ordered Diffusion Message Passing

Use a learned scalar ordering function to turn a symmetric local Gaussian graph kernel into a directed, row-stochastic message-passing operator. The asymmetric tilt lets neighboring nodes communicate preferentially along an inferred dynamical direction, while the Gaussian factor retains locality and diffusion-like smoothing.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Ordered Diffusion Kernels arXiv:2608.18019
△ Mechanism confirmed, baseline not beaten 2026

Dimension-Free Brenier Transport Layer

Build a neural transport layer by parameterizing a convex potential whose gradient maps a semi-log-concave latent distribution into a compact convex data domain. Use the paper's dimension-free Lipschitz certificate to set the layer's Jacobian scale, initialize the potential, and reject or regularize parameter updates that create excessive curvature. The goal is a bounded-output transport module that is less sensitive to latent dimension than diameter-based spectral heuristics.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Dimension-Free Lipschitz Bounds for Brenier Maps to Compactly Supported Log-Concave Targets arXiv:2608.15906
△ Mechanism confirmed, baseline not beaten 2026

Exact-boundary geometric PINN ansatz

Replace penalty enforcement of boundary and asymptotic conditions by a neural-network parametrization that satisfies them identically. For a disc mapped to hyperbolic half-space, construct the positive height coordinate with a vanishing boundary factor and construct tangential coordinates as a fixed boundary extension plus a vanishing correction.

Useful7/10
Difficulty4/10
Novelty6/10
Paper: A user's guide to PINNs in geometric analysis: lessons from the asymptotic Plateau problem arXiv:2607.28733
△ Mechanism confirmed, baseline not beaten 2026

Feasible Action Mapping Safety Layer

Let a neural policy emit an unconstrained abstract action z, then solve a state-dependent feasibility problem that maps z to an admissible optimal-control parameter p before execution. Unlike coordinate-wise clipping, the mapping accounts for predicted dynamics, coupled state and input constraints, and recursive feasibility, allowing the policy to retain a simple unconstrained output space while the controller enforces plant constraints.

Useful7/10
Difficulty6/10
Novelty5/10
Paper: Bridging Reinforcement Learning and Optimal Control via Feasible Action Mapping arXiv:2607.23930
△ Mechanism confirmed, baseline not beaten 2026

Tangent-Branch Neural Evasion Layer

Wrap a neural multi-agent policy with an analytic planner that generates turn-straight trajectories tangent to pursuer surveillance disks, then selects the branch with the smallest predicted completion time. The network supplies high-level preferences or residual corrections, while the geometric layer prevents unnecessarily entering exclusion regions and exposes an explicit branch-switching signal for training.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Semi-Explicit Solutions to the Prying-Pedestrian Surveillance-Evasion Differential Game and Extensions to Two Pursuers arXiv:2607.21087
△ Mechanism confirmed, baseline not beaten 2026

Conditional OT barycenter feature augmentation

Construct synthetic latent examples from an optimal-transport barycenter of several source domains, restricting the barycentric mass to the context region relevant to the prediction. The resulting representations preserve cross-source consensus while reducing domain-specific nuisance variation. Train on the original examples plus barycentric latent examples with transported soft labels.

Useful7/10
Difficulty5/10
Novelty4/10
Paper: Harnessing Heterogeneous Data for Conditional Optimization via Optimal Transport arXiv:2607.19761
✗ Failed on benchmark 2026

Doubled-angle orientation order pooling

Add a differentiable orientation-pooling layer after steerable filters or an orientation-bin expansion. It aggregates unoriented line evidence using doubled-angle vectors, so a feature at angle θ is identical to one at θ+π, while symmetric orientations cancel naturally instead of producing an arbitrary mean angle. Feed the network both the Cartesian order parameter and its magnitude-based confidence.

Useful7/10
Difficulty4/10
Novelty6/10
Paper: Perceived vertical and eye level as one orientation order parameter: a closed-form account of the Li-Matin rules for egocentric space arXiv:2607.19681
△ Mechanism confirmed, baseline not beaten 2026

OT-Sufficient Bottleneck Flow Matching

Replace a conventional regression bottleneck with an encoder whose representation is trained to preserve the conditional law of the target through conditional optimal transport. The encoder produces a low-dimensional z, while a conditional velocity field transports a fixed reference distribution into the observed target distribution given z; minimizing flow-matching error forces z to retain multimodality, conditional variance, and other distributional information.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Learning sufficient low-dimensional structures through conditional optimal transport arXiv:2607.18861
✓✓ Beats tuned baseline 2026

Symmetry-Quotiented Local Correlation Encoder

Replace raw molecular orientation vectors with local scalar features invariant under common three-dimensional rotations and the apolar transformation u_i -> -u_i. Feed these channels to a CNN autoencoder, VAE, or contrastive encoder so that configurations on the same physical symmetry orbit have identical inputs or latent codes. This should improve unsupervised phase discovery without supplying order-parameter labels.

Useful7/10
Difficulty4/10
Novelty5/10
Paper: Representation-Dependent Machine Learning of the Isotropic-Nematic Transition in the Lebwohl-Lasher Model arXiv:2607.16481
△ Mechanism confirmed, baseline not beaten 2026

Harmonic-coordinate neural PDE ansatz

Build a complex-valued coordinate map q(x) whose components are harmonic and whose gradients are mutually null, then feed q(x) into an otherwise unconstrained neural function v. Any learned output of the form u(x)=v(q(x)) is analytically harmonic when the constraints are satisfied, so the network does not need to rediscover the Laplace structure from collocation data. This is especially suitable for two-dimensional elliptic PDEs, where q=x+iy is the canonical example.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Harmonic Variables for Laplace Operators on Homogeneous Spaces arXiv:2607.14132
✗ Failed on benchmark 2026

ZCA In-Context Output Transport

Train a neural surrogate to predict outputs in a source-domain ZCA-whitened space, then adapt to a shifted domain using only the shifted domain's output mean and covariance. At inference, transport the network prediction through the target covariance square root, yielding a weight-free correction that preserves output-coordinate semantics and can be applied to MLP, CNN, graph-NN, or transformer regressors.

Useful7/10
Difficulty4/10
Novelty5/10
Paper: Gradient-Free Topology Adaptation for Power Flow Surrogates via In-Context Whitening arXiv:2607.12241
✗ Failed on benchmark 2026

Haar-Averaged Invariant Attention

Replace ordinary pairwise attention similarity by an affinity averaged over transformed keys or values. The resulting attention is invariant to the group action on either input and avoids requiring the network to learn identical attention patterns for every rotated or transformed copy.

Useful7/10
Difficulty5/10
Novelty5/10
Paper: Group Invariant Spectral Embedding arXiv:2607.08987
✗ Failed on benchmark 2026

Cross-domain amortized inverse operator

Use separate measurement-domain and target-domain token sets so a network can infer a field on one spatial domain from sparse observations on another in one forward pass. The same decoder can answer arbitrary target query points, avoiding an optimization loop for each inverse instance.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: GAIA: Geometry-Adaptive Operator Learning for Forward and Inverse Problems arXiv:2607.01128
Unverified 2026

Bicomplex Spatio-Temporal Residual Encoder

Replace independent per-sensor temporal encoders with a shared two-dimensional transform whose temporal and spatial phase channels remain algebraically distinguishable. Train a neural residual detector and localization head on the transformed coefficients, forcing perturbations with the same temporal signature but different positions to differ primarily in their spatial phase.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Multivariable Geometric Laplace Transform and Fault Detection in Distributed-Converter Lines arXiv:2609.02431
Unverified 2026

Dimension-calibrated contractive branch attractor

Construct a recurrent or generative network from finitely many contractive branches whose hidden-state attractor has a prescribed similarity dimension. The branch contraction ratios determine the target complexity through the equation sum_i r_i^s = 1, while a separation penalty approximates the open set condition and prevents branch collapse or excessive overlap.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: The Moran--Hutchinson formula in semimetric spaces arXiv:2608.16817
Unverified 2026

Adaptive spectral-gap toroidal encoding

Replace ordinary absolute positional embeddings with coordinates on a learned flat torus and use dual-lattice Fourier characters as positional features. Control the covariance of the coordinate fundamental domain so that the paper's inequality guarantees a lower bound on the smallest nonzero positional frequency, preventing the learned periodic coordinate system from developing arbitrarily weak or nearly constant modes.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Spectral and Isoperimetric Bounds on Flat Tori arXiv:2608.13052