Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

△ Mechanism confirmed, baseline not beaten 2026

Equivariant Shared-Mechanism World Model

Use the paper's families of local graph embeddings to identify repeated occurrences of the same causal substructure across time steps, environments, or entities. Feed every aligned occurrence through one shared transition mechanism and impose an explicit equivariance penalty under the symmetry group acting on occurrence indices, rather than learning an independent predictor for every context.

Useful8/10
Difficulty5/10
Novelty5/10
Paper: Symmetries and Causality: Causal Effect Identification Beyond IID Data arXiv:2609.03697
△ Mechanism confirmed, baseline not beaten 2026

Exact-Jacobian Flow Controller

Replace an unconstrained trajectory or density network with a stack of RealNVP-style triangular coupling layers whose inverse and log-volume change are analytic. Condition the coupling subnetworks on the task prompt and time, so the same invertible module represents task-specific population states while providing an exactly computable density and score surrogate.

Useful7/10
Difficulty5/10
Novelty4/10
Paper: Self-supervised In-context Operator Learning for Stochastic Mean-Field Control arXiv:2608.18282
✗ Failed on benchmark 2026

Signature-conditioned cylindrical law head

Add a conditional-law head that maps a compact representation of an initial distribution and a shared-noise trajectory to a Gaussian mixture, then computes downstream predictions as analytic expectations under that mixture. This can replace expensive particle rollouts or particle pooling in stochastic world models and conditional diffusion systems while retaining multimodality.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: A cylindrical neural approximation theorem for conditional laws of McKean-Vlasov equations with common noise arXiv:2608.08040
△ Mechanism confirmed, baseline not beaten 2026

Non-Gaussian Perron–Frobenius Latent Filter

Replace Gaussian covariance propagation in a neural state-space model with a finite Perron–Frobenius operator acting on coefficients of a learned density basis. A neural encoder maps observations to latent states, while an eDMD-derived matrix transports the full coefficient vector and supports multimodal or skewed uncertainty. This creates a cheap deterministic uncertainty layer that can be rolled forward for long horizons without repeatedly sampling particles.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: An extended Perron-Frobenius operator filter for nonlinear state estimation arXiv:2607.26632
△ Mechanism confirmed, baseline not beaten 2026

Conditional-copula probabilistic head

Replace a generic multivariate Gaussian or independently factorized output head with separate marginal quantile models and a conditional copula module. The marginals determine each output's calibrated one-dimensional distribution, while the copula models dependence on the uniformized variables, allowing the network to represent asymmetric correlations and tail co-movement without forcing a particular marginal family.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Conditional copula representations and extremal bounds for multivariate statistical functionals arXiv:2607.26256
✗ Failed on benchmark 2026

Causal-Masked Neural SDE

Replace a fully connected neural SDE drift with coordinate-wise functions that can read only the paths of graph parents. Learn soft edge gates and penalize violations of the paper's pathwise Lipschitz condition, so the model remains stable during long rollouts and supports explicit interventions on selected coordinates.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Causal Graphs, Markov Properties and Do-calculus for Stochastic Differential Equations arXiv:2607.12140
✓✓ Beats tuned baseline 2026

Multiplicative Manifold Unscented Recurrent Cell

Replace the Euclidean hidden-state update of a recurrent or state-space neural network with a mixed manifold state containing a rotation component and Euclidean features. Propagate uncertainty with sigma points in tangent error coordinates, retract rotational perturbations through the exponential map, and compute the training loss from the predicted covariance. This avoids invalid rotations and captures second-order curvature effects that a first-order EKF-style recurrent cell misses at large…

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Fully Multiplicative Attitude and Orbit Determination for Deep space Navigation arXiv:2607.10072
Audited (legacy) 2026

Joint-Particle Distributional Critic

Replace independent per-action distributional value heads with a critic whose shared latent particle produces a vector of return samples for all actions simultaneously. Train the predicted joint return vector against a Bellman target vector formed from coupled counterfactual reward-transition samples, using a sliced Wasserstein loss. The greedy action is selected by the mean of the corresponding marginal particles, while shared particles retain cross-action dependence for learning and…

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Learning to Control Coupled-Dynamics Environments with Joint Markov Decision Processes arXiv:2608.22765
Unverified 2026

Resolution-Invariant Hilbert-Space CNF Prior

Replace a finite-dimensional latent prior for fields or trajectories with a continuous flow acting on function coefficients in a separable Hilbert space. A Gaussian reference with an explicit covariance spectrum is transformed by a learned vector field, so the model can represent smooth, non-isotropic function distributions while sharing parameters across discretization resolutions.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Learning Informative Prior with Infinite-Dimensional Continuous Normalizing Flow for Bayesian Inverse Problem arXiv:2609.03343
Unverified 2026

Copula Correction Layer for Joint Predictive Calibration

Add a post-processing or differentiable output layer that first matches each predicted marginal distribution to the data marginal and then corrects the remaining joint dependence by transporting the predicted copula to the observed copula. This directly targets the discrepancy that sequential marginal or quantity-of-interest constraints cannot remove.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Copula Transformations for Data-Consistent Inversion arXiv:2609.02832
Unverified 2026

Positive Reflected Bellman Layer

Replace an unconstrained spatial aggregation in a neural PDE surrogate or controlled-dynamics model with a fixed-branch expectation layer. Each output is a maximum over controls of a nonnegative weighted average of next-state values, with reflected overshoots attenuated by Robin factors. Increasing any input value therefore cannot decrease the output, giving a hard monotonicity and positivity property instead of relying on a penalty.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: A Positivity-Preserving Expectation Scheme for Hamilton--Jacobi--Bellman Equations with Oblique Robin Boundary Conditions arXiv:2608.11936
Unverified 2026

Measure-Lifted Entropy Amplifier

Replace a deterministic latent state with a probability measure over latent states, represented by particles or weighted prototypes. Apply the learned latent transition to every particle, so one base trajectory map induces a dynamics on distributions; use an entropy-preservation or entropy-growth regularizer to prevent collapse of the ensemble. The mechanism predicts that any positive base-state trajectory entropy can generate unbounded distinguishability in the ideal measure space through…

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Entropies of compact subsets and supported measures arXiv:2608.09702
Unverified 2026

Wasserstein-Gated Cached Equilibrium Adapter

Add a small constrained equilibrium layer whose response depends on the current neural state and recent exogenous history, then cache responses keyed by a learned history embedding. For a new history, reuse a cached response only when an empirical Wasserstein distance to the cached history is below a threshold; otherwise run a few inner optimization iterations. The paper's local Holder and trajectory-stability results motivate graceful degradation rather than catastrophic errors for nearby…

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Stability of Differential Stochastic Variational Inequalities with History-Dependent Responses and Transfer Learning arXiv:2608.06923
Unverified 2026

Random-Batch Interaction Layer

Replace an all-pairs interaction or attention-like message field with an unbiased estimator obtained from a uniformly random partition into batches. Keep the partition fixed for \(\tau\) integration steps, then redraw it; the paper's error law predicts that too-long reuse causes a quadratic-in-\(\tau\) error accumulation, while finite-population error decreases as \(N^{-1}\).

Useful6/10
Difficulty4/10
Novelty5/10
Paper: Mean field error estimate of the random batch method for vortex blob dynamics for the 2D Navier--Stokes Equation arXiv:2608.06533
Unverified 2026

Automaton-Gated Objective Switching

Augment a neural policy with deterministic DFA states for the task objective and safety constraint, then select among objective-specific policy heads using those states. Before either target is reached, execute a mixed policy; after one target is reached, switch permanently to the policy specialized for the remaining target.

Useful6/10
Difficulty4/10
Novelty5/10
Paper: Optimal Constrained sc-LTL Planning in MDPs via Switching Policies arXiv:2608.05021
Unverified 2026

Random-start learned AMP

Construct an unrolled phase-retrieval network that begins with an isotropic Gaussian estimate rather than a spectral initializer. Retain the AMP residual correction and Onsager subtraction, but learn the scalar measurement denoisers and step sizes; use several random starts and select the iterate with the lowest measurement residual.

Useful6/10
Difficulty5/10
Novelty4/10
Paper: Approximate Message Passing with Random Initialization for Phase Retrieval arXiv:2608.01654
Unverified 2026

Contracting Countable-Branch Router

Construct a routed neural state update from a collection of branch maps whose inverse-style refinement operators are uniformly contractive. The discrete routing sequence acts as an expandable symbolic code, while contraction makes the continuous state associated with a long routing history insensitive to initialization and earlier perturbations. Use a finite active top-\(K\) set during training, but retain an expandable branch table so the model can represent increasingly complex or rare modes.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Infinite-Piecewise Expanding Maps: Chaos, Ergodicity and Invariant-Set Complexity arXiv:2608.00398
Unverified 2026

Reactive Mass-Weighted Message Passing

Augment every graph or set token with a positive learned mass M_i that controls how strongly it contributes to other nodes and evolves through a growth-minus-inhibition equation. Use separate learned interaction kernels for state transport and mass inhibition, while retaining a directed interaction matrix so the layer is not forced to be permutation-symmetric or conservative.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: A note on application of mean-field limit to non-exchangeable non-conservative systems arXiv:2607.20014
Unverified 2026

Hashed Local-Density Particle Layer

Replace an O(N^2) kernel-density interaction in a particle neural SDE or diffusion sampler with a clipped, randomly shifted histogram density estimate. Feed the local estimated density into the particle drift as a multiplicative gain, preserving density-dependent dynamics while evaluating all particles through occupied-cell hashing in expected O(N) time for fixed dimension and number of shifts.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Density-Dependent McKean--Vlasov Diffusions: Subgaussian Occupancy Bounds and Polynomial Propagation of Chaos arXiv:2607.19583
Unverified 2026

Saturating Trail Memory for Asynchronous Multi-Agent Networks

Equip multiple recurrent agents with a shared spatial or token-level trail field whose influence is a bounded function of accumulated visitation, rather than an unbounded additive memory. Use the paper's simultaneous/sequential invariance as a falsifiable design target: parallel and randomly ordered asynchronous agent updates should produce nearly identical predictions when trail occupancy is saturated, while deliberately nonsaturating controls should show order dependence. This can enable…

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Exact collective first-passage statistics of N trail-interacting walkers arXiv:2607.13213
Unverified 2026

Collider-Aware DAG Variational Network

Replace independent uncertainty heads in a branching neural network with a structured variational posterior whose non-root node distributions condition on jointly sampled latent states of all parents. This allows collider evidence to explain away upstream uncertainty: evidence at a child can alter the posterior over several parent branches instead of leaving their uncertainties artificially independent. The approach can be implemented as a stochastic DAG network and trained with an evidence…

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Deep Gaussian Processes on Directed Acyclic Graphs arXiv:2607.09645
✗ Mechanism failed 2026

Certainty-Equivalent Auxiliary Critic

For risk-sensitive or recursive objectives, add a separate network that predicts the conditional certainty equivalent of the next-state continuation value, rather than forcing the value network to approximate a nested nonlinear expectation directly. Train the value, policy, and certainty-equivalent heads with Bellman and first-order residuals jointly.

Useful6/10
Difficulty4/10
Novelty6/10
Paper: Deep Learning for Dynamic Programming with Recursive Utility Using First-order Conditions arXiv:2607.09461
✗ Mechanism failed 2026

Thermodynamic Two-State Expert Gate

Add a slow latent two-state gate to a recurrent, state-space, or world-model network so that separate experts represent two qualitatively different dynamical regimes. Train the gate using the paper's two-state population and fluctuation mechanism rather than allowing an unconstrained softmax to average incompatible regimes. The model should allocate extra capacity near the gate's susceptibility peak, where regime uncertainty and forecast variance are predicted to be largest.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Structural Origin of Water Heat Capacity Anomaly from Classical and Quantum Simulations arXiv:2607.08957
Unverified 2026

Gaussian-resummed sparse interaction layer

Replace a dense weak pairwise interaction over tokens or graph nodes by an analytically resummed Gaussian quadratic term plus explicitly evaluated corrections containing only subsets of size at most R. Choose R from the target approximation error and interaction scale instead of fixing the interaction order heuristically. This is intended for dense global attention, graph interactions, or latent-variable energy models with individually small couplings.

Useful5/10
Difficulty7/10
Novelty8/10
Paper: Diffuse Gaussian Truncation For Deterministic Approximate Counting arXiv:2609.04079