Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

△ Mechanism confirmed, baseline not beaten 2026

Equivariant Shared-Mechanism World Model

Use the paper's families of local graph embeddings to identify repeated occurrences of the same causal substructure across time steps, environments, or entities. Feed every aligned occurrence through one shared transition mechanism and impose an explicit equivariance penalty under the symmetry group acting on occurrence indices, rather than learning an independent predictor for every context.

Useful8/10
Difficulty5/10
Novelty5/10
Paper: Symmetries and Causality: Causal Effect Identification Beyond IID Data arXiv:2609.03697
✓✓ Beats tuned baseline 2026

Adaptive Physics-Lifted Koopman State Space

Replace a purely nonlinear recurrent transition with a learned observable map followed by an explicitly linear latent evolution model. Include the original latent state and a small set of nonlinear observables, and update the linear transition online with forgetting-factor recursive least squares when the environment or task dynamics change.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: Physics-based Online Adaptive Koopman Model Predictive Attitude Control for Combined Spacecraft with Dynamic Uncertainties arXiv:2609.02534
△ Mechanism confirmed, baseline not beaten 2026

Seed-Anchored Budgeted Graph Context

Replace arbitrary graph serialization or global top-k retrieval with deterministic locality tiers centered on entities matched by the question. Render every candidate unit in the highest-priority seed-local tiers before admitting more distant or weakly connected material, and use stable identifiers to make ties reproducible. If the complete seed-local candidate region fits within the context budget, no relevant unit in that region is lost to truncation.

Useful8/10
Difficulty4/10
Novelty7/10
Paper: Seed-Anchored Budget-Bounded Graph Rendering for Question Answering on Industry-Standard Power-Grid Information and Exchange Models arXiv:2609.02011
✗ Mechanism failed 2026

Fused truncated-power KAN activation

Replace Cox-de Boor evaluation of each cubic B-spline edge activation with its fixed truncated-power expansion. Normalize each scalar edge input to a bounded knot coordinate, evaluate the five shifted cubic positive-part terms in parallel, and contract them with the learned spline coefficients inside one fused kernel.

Useful8/10
Difficulty4/10
Novelty7/10
Paper: FlashKAN: B-Spline KANs via Truncated Power Form arXiv:2609.01956
✗ Failed on benchmark 2026

Differentially Passive Neural Blocks

Replace selected residual, recurrent, or state-space blocks by modules whose input-output Jacobians satisfy an IODP inequality throughout a prescribed activation domain. The constraint controls incremental amplification between two trajectories without requiring either trajectory to remain near one fixed equilibrium, so it should improve robustness to changing contexts and prevent exploding long-horizon sensitivities.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: Decentralized and Equilibrium-Set-Oriented Stability Analysis and Control for Power Systems arXiv:2609.00497
✗ Failed on benchmark 2026

Role-Filler Attention

Replace dense attention over structured object tokens with attention over role-filler tensor-product representations. A learned query specifies both a role and a filler, retrieves objects matching that binding, extracts a target role, and rebinds the extracted filler into an output object.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: TPR-Attention for Combinatorial Generalization arXiv:2608.30124
△ Mechanism confirmed, baseline not beaten 2026

Partial-ReNoise Neural Architecture Mutation

Replace independent architecture generation with a diffusion mutation kernel that starts from a known valid neural architecture, re-noises it for only a fraction of the diffusion horizon, and denoises it conditionally toward a new architecture. The resulting candidates should remain closer to the parent and retain validity at low mutation strength, while larger re-noising fractions should produce greater novelty and access to distinct architectural basins.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: From Generation to Discovery: Diffusion Mutation Kernels for Circuit and Physical Design arXiv:2608.27649
✗ Mechanism failed 2026

Consensus-Corrected Topology-Invariant GNN

Replace ordinary topology-sensitive message passing with scalar-gated aggregation followed by an explicit correction that aligns local node states with a graph-wide consensus component. The correction should make node embeddings less sensitive to line or edge removals while preserving local information needed for prediction. This is suitable for graph neural networks and graph-based world models exposed to changing graph sizes or sparsity patterns.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: UNION: A Unified AC-OPF Framework for Topology-Varying Real-Time Grid Operation arXiv:2608.25784
△ Mechanism confirmed, baseline not beaten 2026

Farkas-Certified Neural Safety Shield

Insert a constraint-reduction layer between a policy network and its executed action. The policy proposes an action, while the layer retains only geometrically extreme collision and obstacle constraints and verifies that every discarded halfspace is implied by the retained ones through nonnegative conic multipliers. The reduced projection or quadratic program is therefore equivalent to the full tightened safety filter whenever certification succeeds, but uses substantially fewer constraints.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Scalable Tube-Tightened Multi-Agent Safety via Certified Constraint Reduction arXiv:2608.25323
✗ Failed on benchmark 2026

RG Pyramid Flow Matching

Replace a full-resolution global flow-matching or diffusion model with a hierarchy of local velocity fields operating on progressively finer grids. Coarse levels generate long-wavelength structure and pass it to fine levels through upsampling and residual conditioning, while every velocity network uses only a locality radius prescribed by the RG bound. This should preserve long-range correlations without quadratic global attention or a full-resolution global receptive field.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: Renormalization Group Flow Matching for Scalable Local Generative Modeling arXiv:2608.23696
✗ Failed on benchmark 2026

Non-Abelian Event-Order Memory

Augment an RNN or state-space model with a three-dimensional auxiliary spin updated by noncommuting rotations associated with event types or token classes. The ordered product preserves information that additive counters discard: two sequences with the same number of each event can produce different final spins when their event order differs. Train the spin axes, angles, and readout jointly with the task model while constraining every update to remain on the sphere.

Useful8/10
Difficulty5/10
Novelty8/10
Paper: Non-Abelian Spin Counting of Ordered Stochastic Trajectories: Reentrant Finite-Time Chern Numbers arXiv:2608.23533
△ Mechanism confirmed, baseline not beaten 2026

Delay-Aware Plug-and-Play Residual Capacity

Construct a residual network from independently attachable modules, but permit only a number of modules whose aggregate feedback gain lies inside a delay-dependent admissible interval. Estimate deployed end-to-end latency and each module's local Jacobian gain, then reject or bypass additional modules when the predicted delayed-loop stability boundary is crossed. This turns variable-width or depth scaling into a falsifiable control problem rather than an empirical choice.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Admissible Unit Range of Plug-and-Play Distributed Energy Resource (DER) Systems Under Delay: A Scalable Design Framework arXiv:2608.23328
✓✓ Beats tuned baseline 2026

Resolution-normalized Hilbert dictionary

Build a shallow neural model whose input at every discretization level is embedded into a common Hilbert space with uniformly bounded norm, and constrain every neuron parameter in the corresponding dual norm. The statistical complexity then depends on the Hilbert norm bound rather than the number of retained coordinates, allowing one model design to operate across increasingly fine measurements.

Useful8/10
Difficulty4/10
Novelty7/10
Paper: Resolution-Consistent Greedy Neural Approximation on Infinite-Dimensional Spaces arXiv:2608.20812
✗ Mechanism failed 2026

Reference-Preserving Martingale Layer

Replace an unconstrained stochastic transition between categorical or discretized latent distributions by a transition matrix that preserves a prescribed reference distribution while mapping relative populations through a martingale. This prevents the layer from inventing arbitrarily sharp deviations from the reference and imposes a convex-order monotonicity condition on uncertainty across layers or diffusion time steps.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: State convertibility and fluctuation theorems from a dynamical reference: majorization meets martingales arXiv:2608.19391
✓✓ Beats tuned baseline 2026

Conservative Flux Neural Operator

Replace pointwise prediction of the next field with prediction of a learned flux followed by a discrete divergence. Combine Fourier spatial mixing with a causal temporal kernel over the recent resolved-history slab, so the model learns finite-memory closure effects while preserving local conservation exactly under periodic or compatible boundary conditions. The architecture should reduce spurious mass drift and improve autoregressive rollout stability on coarse-grained PDE data.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: Flux-form spatiotemporal neural operators for coarse-grained dynamics of multiscale PDEs arXiv:2608.18148
△ Mechanism confirmed, baseline not beaten 2026

Critical-Region LP Policy Layer

Replace black-box differentiation through an embedded LP decision with an analytic Jacobian computed from the LP’s active basis. A neural policy emits LP coefficients or right-hand sides; the LP returns the decision, while the backward pass uses the basis inverse and dual sensitivity, avoiding solver unrolling and finite-difference noise.

Useful8/10
Difficulty5/10
Novelty5/10
Paper: Simulation-Optimization of Systems of Optimizers: Exploiting the Inner Optimization's Geometry arXiv:2608.18129
△ Mechanism confirmed, baseline not beaten 2026

Hadamard-CLIP joint interaction head

Replace the usual sum of pairwise modality similarities with a higher-order score based on the coordinatewise Hadamard product of all normalized modality embeddings. For modalities indexed by i=1,...,m, score a tuple using s(x_1,...,x_m)=\omega^\top(\bar g_1(x_1)\odot\cdots\odot\bar g_m(x_m)), where \omega is learned and \odot is coordinatewise multiplication. This adds explicit m-way interactions without concatenating raw features or introducing a joint encoder.

Useful8/10
Difficulty4/10
Novelty7/10
Paper: Expressivity In Multimodal Contrastive Learning arXiv:2608.17203
✗ Failed on benchmark 2026

Gradient-Flow Commutator Network

Build a neural ODE or invertible transformation whose primitive layers are flows of learned gradient vector fields, then synthesize non-gradient directions using short Lie-bracket commutator products. The paper's bounded-bracket-generation result predicts that restricted gradient primitives can approximate a much larger class of diffeomorphisms than a plain stack of gradient flows.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: The Holonomy of Optimal Mass Transport: The Smooth Case arXiv:2608.15585
△ Mechanism confirmed, baseline not beaten 2026

Differentiable Asymmetric Admissibility Layer

Replace hard clipping or post-hoc asymmetric saturation with a dynamic output state that remains inside a prescribed asymmetric interval. A neural network emits a command uc, while the realized output u evolves through the APIR vector field, producing bounded actions, temporal smoothing, and gradients that remain available in the interior.

Useful8/10
Difficulty4/10
Novelty7/10
Paper: Admissibility-Preserving Control for Strict-Feedback Nonlinear Systems with Asymmetric Actuator Constraints arXiv:2608.15375
✗ Failed on benchmark 2026

Ghost-State Adaptive Recurrent Cell

Replace a single recurrent state update with fast feature relaxation, activity evolution, and a slow adaptive state that modulates the activity vector field. Tune the activity subsystem near a controllable saddle-node so that it retains a useful transient regime for a predictable number of steps, enabling delayed switching and long-horizon memory without requiring a large hidden state.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Ghost Dynamics in Receptor Signalling Networks: A Fast--Slow Adaptive Extension of Competitive Cancer Inhibition Models arXiv:2608.15300
△ Mechanism confirmed, baseline not beaten 2026

Diagonalizable Directed Message Passing

Replace arbitrary directed-edge weights in a graph neural ODE or recurrent message-passing layer by weights constructed to make the directed Laplacian diagonalizable. This removes Jordan-block coupling, allowing the linearized graph dynamics to be represented as independent eigenmodes rather than modes with polynomial transients such as t^k exp(lambda t).

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Positive Arc-Weight Design Makes Every Directed Laplacian Diagonalizable arXiv:2608.14439
✗ Failed on benchmark 2026

Conditioned Irregular-Delay State Encoder

Replace uniformly spaced history taps in a neural state-space encoder with a fixed or learned set of non-uniform delays. Regularize the resulting delay-observation matrix to have a large smallest singular value, which makes latent-state reconstruction less sensitive to irregular timestamps and observation noise. This is directly applicable to event-based data, missing timestamps, and systems with multiple time scales.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Stable Takens' Embedding Theorem for Non-Uniformly-Sampled Linear Systems arXiv:2608.14001
✓✓ Beats tuned baseline 2026

Equivariant Variational Field Network

Represent a scalar energy or free-energy functional of a three-dimensional neural field using translation- and rotation-equivariant convolutions, and produce the field prediction by minimizing the total functional rather than by a direct decoder. The same functional can then generate equilibrium states, forces, and response observables under new external fields, resolutions, and system sizes.

Useful8/10
Difficulty7/10
Novelty6/10
Paper: Equivariant learning of a transferable three-dimensional classical density functional arXiv:2608.13506
✓✓ Beats tuned baseline 2026

Nonlinearity-Subtracted Latent State-Space Model

Build a latent continuous-time neural model with dynamics \(\dot{z}=Az+f_\phi(z)\), where \(f_\phi\) is known, separately computed, or frozen, and \(A\) is learned exclusively from the derivative residual after subtracting \(f_\phi(z)\). Parameterize \(A\) with a truncated SVD or low-rank factorization so its eigenvalues directly predict local stability and long-horizon growth.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Data-driven linear analysis of dynamical systems via nonlinearity-subtracted dynamic mode decomposition arXiv:2608.13373