Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

△ Mechanism confirmed, baseline not beaten 2026

Equivariant Shared-Mechanism World Model

Use the paper's families of local graph embeddings to identify repeated occurrences of the same causal substructure across time steps, environments, or entities. Feed every aligned occurrence through one shared transition mechanism and impose an explicit equivariance penalty under the symmetry group acting on occurrence indices, rather than learning an independent predictor for every context.

Useful8/10
Difficulty5/10
Novelty5/10
Paper: Symmetries and Causality: Causal Effect Identification Beyond IID Data arXiv:2609.03697
✓✓ Beats tuned baseline 2026

Resolution-normalized Hilbert dictionary

Build a shallow neural model whose input at every discretization level is embedded into a common Hilbert space with uniformly bounded norm, and constrain every neuron parameter in the corresponding dual norm. The statistical complexity then depends on the Hilbert norm bound rather than the number of retained coordinates, allowing one model design to operate across increasingly fine measurements.

Useful8/10
Difficulty4/10
Novelty7/10
Paper: Resolution-Consistent Greedy Neural Approximation on Infinite-Dimensional Spaces arXiv:2608.20812
✗ Mechanism failed 2026

Reference-Preserving Martingale Layer

Replace an unconstrained stochastic transition between categorical or discretized latent distributions by a transition matrix that preserves a prescribed reference distribution while mapping relative populations through a martingale. This prevents the layer from inventing arbitrarily sharp deviations from the reference and imposes a convex-order monotonicity condition on uncertainty across layers or diffusion time steps.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: State convertibility and fluctuation theorems from a dynamical reference: majorization meets martingales arXiv:2608.19391
✗ Mechanism failed 2026

Permutation-family residual network

Replace direct learning of a highly cancelling signed observable with a quotient-space model over symmetry orbits of inputs. Predict a physically constrained baseline for each family and use an LSTM or set/graph encoder only for the residual many-body correlation, then aggregate family predictions with known signed weights instead of forming a noisy sample-level ratio.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Learning the Fermion sign structure in path-integral Monte Carlo arXiv:2607.15060
△ Mechanism confirmed, baseline not beaten 2026

Tail-Aware Verifier Portfolio

Use the paper's tail comparison to decide when another call from the same verifier family is useless and when to switch to a different model, modality, or evidence source. The objective is to reduce the high-alpha survivor population—the incorrect examples that consistently fool one verifier—rather than maximizing average one-shot verifier accuracy.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Partially Correlated Verifier Cascades in LLM Harnesses: Concave Log-Odds, Polynomial Reliability, and Blind-Spot Ceilings arXiv:2607.13918
✗ Mechanism failed 2026

Conditional-information-preserving pooling

Replace ordinary token merging or graph pooling with a learned block map whose output preserves information about a remote target conditioned on the surrounding coarse representation. The paper's majority-spin counterexample gives a concrete failure mode: two microscopic configurations mapped to the same pooled token can imply different predictions for distant variables.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Renormalization group and long-range conditional mutual information in hierarchical models arXiv:2609.02141
△ Mechanism confirmed, baseline not beaten 2026

Mean-field rainbow relation router

Replace independent edge-type logits in a relational graph neural network with a mean-field fixed-point router derived from a colored ERGM. Each edge's color distribution is influenced by its own relation bias and by the expected number of rainbow triangles it forms with neighboring edges, allowing the model to learn coordinated multilayer structures.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Colorful Exponential Random Graph Models arXiv:2608.31130
✗ Failed on benchmark 2026

Critical stochastic min-plus tree layer

Replace deterministic binary-tree pooling or hierarchical feature aggregation by a stochastic merge that chooses either elementwise addition or elementwise minimum. The mixing probability p controls whether zero or sparse states proliferate or disappear, with a predicted absorbing-state transition at p = 1/2.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: Finite-depth scaling and an exact Bernoulli-leaf identity for the min-plus process on the binary tree arXiv:2608.12295
△ Mechanism confirmed, baseline not beaten 2026

GECC-Gated Loop-Aware Message Passing

Construct order-n generalized edges from intersections of local ego-subgraphs and use their overlap statistics to correct ordinary one-hop aggregation. A learned gate should activate the correction only when local generalized-edge closure is high, because dense but internally inconsistent overlaps are precisely where naive loop corrections can become unreliable.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Ensemble-level loopy message passing with generalized-edge closure for percolation arXiv:2608.09397
✗ Mechanism failed 2026

RG Spectral Feature Gate

Replace fixed PCA-rank selection in a hidden layer with a renormalization-group-inspired gate over covariance eigenvalue bands. The gate retains modes whose effective quartic interaction remains unstable or strongly scale-dependent, while pruning bands that flow toward the Gaussian noise fixed point. Unlike top-eigenvalue truncation, this is designed for extensive-rank signal distributed throughout the bulk spectrum.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Data Field Theory: Theory and Applications of the Functional Renormalization Group for Signal Detection arXiv:2607.27236
△ Mechanism confirmed, baseline not beaten 2026

Conditional-copula probabilistic head

Replace a generic multivariate Gaussian or independently factorized output head with separate marginal quantile models and a conditional copula module. The marginals determine each output's calibrated one-dimensional distribution, while the copula models dependence on the uniformized variables, allowing the network to represent asymmetric correlations and tail co-movement without forcing a particular marginal family.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Conditional copula representations and extremal bounds for multivariate statistical functionals arXiv:2607.26256
△ Mechanism confirmed, baseline not beaten 2026

Subcritical Percolation Jordan Readout

Use several independently percolated versions of an input graph and rank nodes by Jordan centrality inside the largest retained components. Feed only the consistently high-ranked nodes to a graph readout or source-localization head, making predictions depend on structurally stable regions rather than shortcut edges or fragile local neighborhoods.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Subcritical percolation and network archaeology on random recursive tree substrate networks arXiv:2607.21428
△ Mechanism confirmed, baseline not beaten 2026

Markov Spectral Equivariant Layer

Replace an orthogonal truncated Fourier or Wigner projection in a compact-Lie-group equivariant layer by a finite-rank Fejér-Markov filter. The filter acts as a normalized positive group convolution, preventing sup-norm amplification and suppressing high-frequency artifacts while retaining exact equivariance.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Structure-Preserving Spectral Dynamic Programming on Compact Lie Groups arXiv:2607.20854
✗ Failed on benchmark 2026

Confidence-Tested LoRA Pruning

Replace deterministic LoRA importance scores with one-sided tests of whether each rank-one update has population contribution at least a user-selected threshold. Maintain empirical contribution samples during fine-tuning, estimate their uncertainty, and prune the components with the weakest statistical evidence while respecting the target rank budget. The method should avoid deleting components merely because their latest minibatch gradient was small.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Statistical Inference for Rank Allocation in Low-Rank Adaptation arXiv:2607.20205
△ Mechanism confirmed, baseline not beaten 2026

Conditional OT barycenter feature augmentation

Construct synthetic latent examples from an optimal-transport barycenter of several source domains, restricting the barycentric mass to the context region relevant to the prediction. The resulting representations preserve cross-source consensus while reducing domain-specific nuisance variation. Train on the original examples plus barycentric latent examples with transported soft labels.

Useful7/10
Difficulty5/10
Novelty4/10
Paper: Harnessing Heterogeneous Data for Conditional Optimization via Optimal Transport arXiv:2607.19761
△ Mechanism confirmed, baseline not beaten 2026

OT-Sufficient Bottleneck Flow Matching

Replace a conventional regression bottleneck with an encoder whose representation is trained to preserve the conditional law of the target through conditional optimal transport. The encoder produces a low-dimensional z, while a conditional velocity field transports a fixed reference distribution into the observed target distribution given z; minimizing flow-matching error forces z to retain multimodality, conditional variance, and other distributional information.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Learning sufficient low-dimensional structures through conditional optimal transport arXiv:2607.18861
✗ Failed on benchmark 2026

Nonlinear Noise-Tightening Drift

Replace a linear restoring drift in score-based sampling, latent dynamics, or stochastic regularization with a state-dependent nonlinear restoring term that is at least as contractive globally and more contractive away from the origin. This should reduce stationary variance without changing the worst-case local contraction certificate.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Wasserstein Stability of Contracting Flows: Effective Rates, Euler Self-Correction, and Noise Tightening arXiv:2607.14291
✗ Failed on benchmark 2026

Causal-Masked Neural SDE

Replace a fully connected neural SDE drift with coordinate-wise functions that can read only the paths of graph parents. Learn soft edge gates and penalize violations of the paper's pathwise Lipschitz condition, so the model remains stable during long rollouts and supports explicit interventions on selected coordinates.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Causal Graphs, Markov Properties and Do-calculus for Stochastic Differential Equations arXiv:2607.12140
Unverified 2026

Zero-Temperature Block-Triangular Routing

Replace an unconstrained attention or MoE routing matrix with a row-stochastic matrix that preserves a learned Gibbs-like distribution over groups. Anneal its temperature so that the router converges to a block-upper-triangular operator: mixing remains flexible within equal-energy groups, while cross-group traffic becomes directional rather than oscillating or collapsing. This creates a hierarchy-aware inductive bias and can reduce the number of active expert or attention connections.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Limits of Stochastic Semigroups and Block-Triangular Majorisation arXiv:2609.04057
Unverified 2026

Probability-coordinate robust likelihood head

Replace a Gaussian regression likelihood with a chart-pulled exponential-family head. The network predicts a natural parameter eta, while each target y is mapped to a bounded probability coordinate u=G(y). Because the sufficient statistic T(u) is bounded, extreme targets cannot create arbitrarily large parameter gradients. The chart determines the tail class, while eta controls the distribution in probability space.

Useful6/10
Difficulty4/10
Novelty6/10
Paper: G-Exponential Families through Probability Coordinates arXiv:2609.03208
Unverified 2026

Copula Correction Layer for Joint Predictive Calibration

Add a post-processing or differentiable output layer that first matches each predicted marginal distribution to the data marginal and then corrects the remaining joint dependence by transporting the predicted copula to the observed copula. This directly targets the discrepancy that sequential marginal or quantity-of-interest constraints cannot remove.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Copula Transformations for Data-Consistent Inversion arXiv:2609.02832
Unverified 2026

Reciprocal Beta-Angle Mixer

Insert a projection-space mixer that combines several fixed or learned directions using reciprocal correlations with the current feature, then normalize the result. The exact construction has a universal beta law for its squared input-output cosine, so it can create controlled angular diversity while remaining deterministic and independent of the chosen direction dictionary.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Universal Beta Incidence Angles: Cauchy Rigidity and Infinite Arrangements arXiv:2609.00603
Unverified 2026

Reciprocal Paired-Density Network

Represent a neural density or feature field by two positive reciprocal branches whose product is the modeled density, analogous to the forward and backward fields in the paper. Add stochastic branching perturbations to the two branches and train their cross-covariance so that the diagonal paired density matches the target while off-diagonal correlations remain finite-range. This creates a structured alternative to an unconstrained single-field uncertainty representation.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Branching stochastic mechanics. I. Clustering and connected correlations within a branching-process representation of the Schrödinger equation arXiv:2608.29807
Unverified 2026

Stable Hyperplane Feature Layer

Replace a dense directional feature map with a finite stable-zonotope layer: a small set of large random vectors creates a hyperplane arrangement over the input sphere, while a cheaper small-jump residual gives continuous features inside each cell. The layer exposes an explicit compute-versus-resolution knob through the jump cutoff and can be used either as an embedding before an MLP or as a lightweight router for mixture-of-experts blocks.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Hausdorff Dimension of the Set of Extreme Points of a Random Countable Stable Zonotope arXiv:2608.28004