Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✓✓ Beats tuned baseline 2026

Equivariant Variational Field Network

Represent a scalar energy or free-energy functional of a three-dimensional neural field using translation- and rotation-equivariant convolutions, and produce the field prediction by minimizing the total functional rather than by a direct decoder. The same functional can then generate equilibrium states, forces, and response observables under new external fields, resolutions, and system sizes.

Useful8/10
Difficulty7/10
Novelty6/10
Paper: Equivariant learning of a transferable three-dimensional classical density functional arXiv:2608.13506
△ Mechanism confirmed, baseline not beaten 2026

MCIS Safety Shield for Neural Controllers

Compute an inner approximation of the states from which a neural controller can keep the plant inside a prescribed safe domain indefinitely, then use the resulting regulation map as a safety shield around the network. At each state, the network proposes an action, but the shield projects or replaces it with an action certified to remain in the invariant set.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: Computing the Maximal Controlled Invariant Set for Neural Network Control Systems arXiv:2608.07908
✓✓ Beats tuned baseline 2026

Power-Balanced Modular Neural Block

Represent each neural module as a Hamiltonian storage system and connect modules through a state-dependent skew or Dirac interconnection instead of arbitrary residual additions. The coupling may change with the hidden state, but its internal power contribution cancels exactly, so total stored energy is controlled only by external inputs and explicitly added dissipation.

Useful8/10
Difficulty5/10
Novelty5/10
Paper: Port-Hamiltonian modelling of coupled rigid/flexible multibody systems arXiv:2608.05143
△ Mechanism confirmed, baseline not beaten 2026

Legendre Feasibility Layer

Replace an unconstrained neural decision head followed by clipping, projection, or penalty losses with a Legendre-regularized optimizer layer that always returns a feasible decision. The layer maps network-produced latent scores smoothly into the relative interior of a convex feasible region and can be differentiated using the inverse Hessian of the regularizer.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: Smooth Learning with Hard Constraints via Legendre-Regularized Policies arXiv:2607.24007
✓ Mechanism works 2026

Transport-Signature Density Decoder

Replace a pixelwise density decoder with a decoder that predicts coefficients of a transport potential relative to a fixed reference density. The reconstructed density is the pushforward of the reference measure through a differentiable transport map, so positivity and total mass are structural properties rather than learned penalties.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: Structure-Preserving Reduced-Order Modeling via Low-Rank Transport Signatures arXiv:2607.01696
△ Mechanism confirmed, baseline not beaten 2026

Parity-block curvature preconditioner

When a model, dataset, or parameterization has an involutive symmetry, transform gradients and curvature into symmetry sectors before applying adaptive updates. The Hessian and optimizer can then be handled as independent even and odd blocks, preventing curvature from one sector from forcing an unnecessarily conservative learning rate in the other.

Useful7/10
Difficulty4/10
Novelty6/10
Paper: The Hessian of Planar Central Configurations in Pair Space: Decomposition, Morse Index and Symmetry Reduction arXiv:2609.01857
✓✓ Beats tuned baseline 2026

Intrinsic Tangent-Projected Point-Cloud Layer

Modify a point-cloud message-passing or neural-operator layer so that scalar gradients, vector features, and vector-to-vector interactions are computed only in the estimated tangent plane of the surface. Projecting both feature values and derivative directions prevents the network from using arbitrary ambient-space normal directions and should improve transfer across differently embedded but intrinsically similar surfaces.

Useful7/10
Difficulty4/10
Novelty5/10
Paper: Solving the Incompressible Navier-Stokes Equations on Oriented Curved Surfaces Discretized by Point Clouds arXiv:2609.00216
✗ Mechanism failed 2026

Orbit-Consistent Equivariant Distillation

Constrain a student policy to transform its action in the same way that the input state is transformed, while constraining its value estimate to remain unchanged. During distillation, augment every teacher-student pair with several symmetry-transformed copies and penalize disagreement after transforming the student action back to the original frame.

Useful7/10
Difficulty5/10
Novelty4/10
Paper: SymVD: Symmetric Vision Language Action Distillation for Robot Manipulation arXiv:2608.29828
✗ Failed on benchmark 2026

Hyperplane-Gated Piecewise Neural Dynamics

Replace a single smooth neural vector field with a finite collection of smooth subnetworks selected by learned affine hyperplanes. The architecture exposes switching geometry directly, allowing it to represent friction-like or threshold dynamics without approximating discontinuities using excessively steep activations.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Learning piecewise-smooth dynamical systems arXiv:2608.19785
✗ Failed on benchmark 2026

Gauge-Covariant Wilson-Loop Regularization

Attach an SU(2) transport matrix to every directed edge of a graph neural network and penalize nontrivial plaquette holonomies instead of penalizing individual edge transformations. The regularizer is invariant to arbitrary local changes of latent representation frame, encouraging path-consistent relational features without requiring all edges to share one global coordinate system.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Frustration without Glass: A Non-Abelian Gauge Model of Network Compatibility arXiv:2608.17817
✗ Mechanism failed 2026

Exact-Curl Neural Field Output

Make a neural network predict a vector potential rather than a magnetic or velocity field, then obtain the physical vector field with a fixed differentiable discrete curl. The reconstructed field satisfies the discrete divergence-free constraint exactly, eliminating divergence-penalty tuning and preventing constraint drift during long rollouts.

Useful7/10
Difficulty4/10
Novelty5/10
Paper: A Structure- and Pressure-Positivity-Preserving Semi-implicit IMEX Finite Volume Scheme for Ideal MHD at All Acoustic Mach and Alfvén Mach Numbers with Generic Equation of State arXiv:2608.15837
✗ Failed on benchmark 2026

Anchored Whitening Layer

Replace a conventional whitening transform with a constrained whitening layer that minimizes cross-channel covariance while requiring every output channel to remain correlated with its designated input channel by at least a threshold \(\rho_{\min}\). The layer exploits the orthogonal freedom in whitening to find a rotation that preserves channel identity instead of arbitrarily mixing features. It can be inserted before an MLP, convolution, or attention projection and compared directly against…

Useful7/10
Difficulty5/10
Novelty6/10
Paper: CORAL: Constrained Oblique Rotation with Anchored Loadings for Fidelity-Constrained Decorrelation arXiv:2608.15319
△ Mechanism confirmed, baseline not beaten 2026

Gram-Whitened Directional Pooling

Represent local feature channels as a smooth directional signal and aggregate them with a partition-of-unity family of learnable spherical atoms instead of hard angular bins. Use the atom Gram matrix to whiten the descriptor and add a projected-energy loss, so the network is rewarded for retaining information in the directional subspace rather than merely producing large correlated channel responses.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Point Feature Descriptor via Directional Partition of Unity on Maps arXiv:2608.12794
✓✓ Beats tuned baseline 2026

Divergence-Free Spherical Kernel Layer

Build a kernel aggregation layer whose output is a tangent vector field on the unit sphere and whose surface divergence is identically zero by construction. For each source point, use a matrix kernel obtained by applying a surface-rotated gradient in the query variable to a scalar zonal kernel; this is a differential-form version of the paper's matrix-valued construction. The layer can replace attention or message passing when the target dynamics are incompressible, such as spherical fluid…

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Divergence-free interpolation of tangential vector fields via matrix-valued kernels arXiv:2608.05547
△ Mechanism confirmed, baseline not beaten 2026

Isometric tensor-network token mixer

Use the relaxed QFT tensor-network topology as a trainable norm-preserving mixer inside a neural block, replacing a dense token-mixing matrix or an expensive global convolution. The network learns data-adapted global interactions while retaining structured O(N log^2 N) application and an exact cheap inverse, making it suitable for image tokens, long sequences, or reversible residual blocks.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Fast Trainable Multilinear Bases for Image Compression arXiv:2608.00053
✓✓ Beats tuned baseline 2026

Symmetry-Preserving Flow Layer

Construct hidden dynamics from permutation-equivariant vector fields and impose antisymmetry through an explicit antisymmetrizing readout. This prevents optimization from learning multiple equivalent copies of the same configuration and makes forbidden symmetry violations exactly zero, rather than merely penalizing them. The design applies to set models, particle systems, graph networks, and architectures handling unordered tokens.

Useful7/10
Difficulty5/10
Novelty4/10
Paper: Spindrift: Learning quantum degeneracy from thermal purity in restricted path integral Monte Carlo arXiv:2607.29590
✗ Mechanism failed 2026

Volume-Threshold Contracting State Layer

Construct a recurrent or state-space layer as a skew product: an expanding bounded feature coordinate drives a linearly contracting hidden state. Constrain the hidden transition matrix A to have spectral radius below one, and monitor the predicted transition ell times the absolute determinant of A equals one: below it, hidden trajectories should occupy a thin or fractal set, while above it they should have substantially higher-dimensional state coverage without losing local contraction.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Geometric Properties of Higher Dimensional Solenoidal Attractors arXiv:2607.27089
✗ Failed on benchmark 2026

Probe-Then-Partitioned Multi-Task Trunk

Train a cheap shared multi-task probe briefly, extract one semantic embedding per task, and use density-based clustering to determine which tasks should share a neural trunk. After clustering, replace the globally shared trunk by one trunk per discovered cluster, with task heads remaining separate; this preserves cooperation among related tasks while isolating destructive task interactions.

Useful7/10
Difficulty5/10
Novelty5/10
Paper: Semantic-Aware Task Clustering for Constructive and Cooperative Multi-Tasking arXiv:2607.21426
△ Mechanism confirmed, baseline not beaten 2026

Invariant-domain learned reconstruction

Insert a neural local reconstruction into a finite-volume or graph-based simulator, but hard-cap its contribution so every reconstructed state remains in the physical admissible set. The network learns accuracy-sensitive gradients or stencil weights; a deterministic limiter, rather than a penalty loss, guarantees positive density and pressure for arbitrary network outputs.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Guarantees by Construction for Learned Finite Volume Schemes on Steady Supersonic Flow arXiv:2607.20171
△ Mechanism confirmed, baseline not beaten 2026

Conditional OT barycenter feature augmentation

Construct synthetic latent examples from an optimal-transport barycenter of several source domains, restricting the barycentric mass to the context region relevant to the prediction. The resulting representations preserve cross-source consensus while reducing domain-specific nuisance variation. Train on the original examples plus barycentric latent examples with transported soft labels.

Useful7/10
Difficulty5/10
Novelty4/10
Paper: Harnessing Heterogeneous Data for Conditional Optimization via Optimal Transport arXiv:2607.19761
△ Mechanism confirmed, baseline not beaten 2026

OT-Sufficient Bottleneck Flow Matching

Replace a conventional regression bottleneck with an encoder whose representation is trained to preserve the conditional law of the target through conditional optimal transport. The encoder produces a low-dimensional z, while a conditional velocity field transports a fixed reference distribution into the observed target distribution given z; minimizing flow-matching error forces z to retain multimodality, conditional variance, and other distributional information.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Learning sufficient low-dimensional structures through conditional optimal transport arXiv:2607.18861
✗ Failed on benchmark 2026

Collision-aware physical-support abstention

Modify a learned sparse encoder so that it distinguishes reliable group activation from unreliable child-ray identity. When test separation information is high but dictionary-orientation information is low, the model should output the active coherent group while abstaining on individual child labels, using a permutation- and sign-invariant support representation.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Honest Physical-Support Inference after Latent Dictionary Learning: Collision Singularities and Minimax Resolution arXiv:2607.16813
✓✓ Beats tuned baseline 2026

Cyclic Momentum Latents

Add a learnable cyclic-coordinate mechanism to latent dynamics so that selected latent coordinates do not enter the Hamiltonian and their conjugate momenta become conserved. This provides an explicit dimensionality-discovery and invariance bias, encouraging the model to represent nuisance or symmetry directions compactly instead of spending independent dynamics capacity on them.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Generating Physically Plausible Parachute Dynamics with Deep Generative Modeling arXiv:2607.12143
△ Mechanism confirmed, baseline not beaten 2026

Electrical Response Attention

Replace unconstrained token-mixing logits by a symmetric zero-row-sum response matrix generated from positive conductances on a small auxiliary electrical network. The resulting mixer has conservation and positivity structure, while circular minors have a prescribed sign pattern associated with positive grove measurements. This is especially suitable for graph neural networks and attention variants that need stable global diffusion rather than arbitrary dense affinities.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Electrical networks, Grassmannians, and cluster algebras arXiv:2607.09975