Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✗ Mechanism failed 2026

Reference-Preserving Martingale Layer

Replace an unconstrained stochastic transition between categorical or discretized latent distributions by a transition matrix that preserves a prescribed reference distribution while mapping relative populations through a martingale. This prevents the layer from inventing arbitrarily sharp deviations from the reference and imposes a convex-order monotonicity condition on uncertainty across layers or diffusion time steps.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: State convertibility and fluctuation theorems from a dynamical reference: majorization meets martingales arXiv:2608.19391
△ Mechanism confirmed, baseline not beaten 2026

Tail-Aware Verifier Portfolio

Use the paper's tail comparison to decide when another call from the same verifier family is useless and when to switch to a different model, modality, or evidence source. The objective is to reduce the high-alpha survivor population—the incorrect examples that consistently fool one verifier—rather than maximizing average one-shot verifier accuracy.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Partially Correlated Verifier Cascades in LLM Harnesses: Concave Log-Odds, Polynomial Reliability, and Blind-Spot Ceilings arXiv:2607.13918
△ Mechanism confirmed, baseline not beaten 2026

Koopman Hankel Dual Autoencoder

Replace pointwise sequence reconstruction with reconstruction of overlapping past and future Hankel windows in a shared latent manifold. A first encoder compresses the delay-coordinate trajectory, while a second decoder or predictor reconstructs the future block from the latent state; training therefore penalizes representations that fit observations but do not preserve dynamical evolution.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Fault detection on manifolds of nonlinear dynamical systems with dual autoencoders arXiv:2608.17698
✗ Failed on benchmark 2026

Anchored Whitening Layer

Replace a conventional whitening transform with a constrained whitening layer that minimizes cross-channel covariance while requiring every output channel to remain correlated with its designated input channel by at least a threshold \(\rho_{\min}\). The layer exploits the orthogonal freedom in whitening to find a rotation that preserves channel identity instead of arbitrarily mixing features. It can be inserted before an MLP, convolution, or attention projection and compared directly against…

Useful7/10
Difficulty5/10
Novelty6/10
Paper: CORAL: Constrained Oblique Rotation with Anchored Loadings for Fidelity-Constrained Decorrelation arXiv:2608.15319
✗ Failed on benchmark 2026

Sensitivity-Conditioned Neural ODE Pruning

Use trajectory sensitivities to remove neural units or parameter groups whose effects are redundant over the available data support. A parameter group is pruned when its Fisher contribution is small or its sensitivity is nearly collinear with other groups, producing a compact neural ODE without relying only on parameter magnitude.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: Identifiability-aware neural ordinary differential equations for parsimonious and reliable dynamic modelling arXiv:2608.13044
✗ Mechanism failed 2026

RG Spectral Feature Gate

Replace fixed PCA-rank selection in a hidden layer with a renormalization-group-inspired gate over covariance eigenvalue bands. The gate retains modes whose effective quartic interaction remains unstable or strongly scale-dependent, while pruning bands that flow toward the Gaussian noise fixed point. Unlike top-eigenvalue truncation, this is designed for extensive-rank signal distributed throughout the bulk spectrum.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Data Field Theory: Theory and Applications of the Functional Renormalization Group for Signal Detection arXiv:2607.27236
△ Mechanism confirmed, baseline not beaten 2026

Noise-Triggered Latent Rank Adaptation

Use the recursive errors-in-variables subspace spectrum as a controller for the width of a latent SSM rather than fixing the state dimension in advance. Neurons or state channels are added when corrected covariance eigenvalues rise above the noise floor and pruned when they remain below it, producing a model-order-adaptive recurrent architecture for nonstationary streams.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: A recursive subspace based method for errors-in-variables model identification of time-varying systems arXiv:2607.17065
Unverified 2026

Stability-calibrated Sinkhorn attention

Replace independently normalized attention or routing weights with an entropic doubly stochastic transport plan, while choosing its regularization ε using the paper's explicit statistical-stability bound. Increase ε when residual inversion or minibatch fluctuations are amplified, and decrease it only when the estimated bound permits sharper assignments.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Uniform Statistical Convergence of Empirical Sinkhorn Potentials with Exponential and Polynomial Dependence on the Regularization Parameter arXiv:2608.29152
Unverified 2026

Layered simplex sparse router

Replace an MoE router's single softmax distribution with a normalized coordinate-wise product of several simplex-valued routing factors. The product preserves positivity and normalization but, as depth grows, concentrates mass on a small subset of experts, creating a mathematically controlled heavy-tailed routing prior rather than relying only on an auxiliary load-balancing loss.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: A Layered Simplex Architecture for Large Alphabets arXiv:2608.19908
Unverified 2026

Measure-Lifted Entropy Amplifier

Replace a deterministic latent state with a probability measure over latent states, represented by particles or weighted prototypes. Apply the learned latent transition to every particle, so one base trajectory map induces a dynamics on distributions; use an entropy-preservation or entropy-growth regularizer to prevent collapse of the ensemble. The mechanism predicts that any positive base-state trajectory entropy can generate unbounded distinguishability in the ideal measure space through…

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Entropies of compact subsets and supported measures arXiv:2608.09702
Unverified 2026

LKJ Covariance for Variational Adapter Blocks

Use an LKJ correlation factor as the correlation component of a variational posterior over a compact adapter, LoRA factor, or Bayesian neural-network parameter block. The model learns marginal scales separately while the correlation matrix remains automatically positive semidefinite and unit-diagonal, avoiding unconstrained covariance matrices, invalid correlations, and fragile covariance decompositions.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Bartlett Couplings of the Onion and Vine LKJ Samplers arXiv:2608.06116
Unverified 2026

Sparse Learnable Power-Law Head

Attach a symbolic sparse head to a neural encoder instead of using a dense final MLP. The head evaluates a library of learnable power-law and interaction terms on nonnegative learned features, jointly optimizes linear coefficients and exponents, and removes inactive terms with coefficient sparsity. This should provide a compact model with better relative-error behavior on positive targets spanning several orders of magnitude.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Discovering Explicit Magnetic Core Loss Equations via Learnable Symbolic Sparse Identification arXiv:2608.00379
Unverified 2026

Gain-aligned branch shunting

Replace an additive nonnegative feature readout by several local divisive branches, where each branch divides a signal pathway by a positive pool chosen to estimate shared multiplicative gain. Initialize or constrain each pool toward the dominant nuisance covariance direction while retaining an additive bypass so the model can reject harmful normalization. This should improve robustness when nuisance gain is shared across features, but not when the pool support is shuffled or its measurements…

Useful6/10
Difficulty5/10
Novelty5/10
Paper: When Branch-Local Shunting Helps: A Gain-Load-Alignment Principle for Dendritic E/I Networks arXiv:2607.24990
Unverified 2026

Mollified Transport-Quantile Layer

Use a transport map \(Q_\theta\) from a fixed latent reference distribution to a data distribution, but expose only its locally averaged version \(\bar Q_{\theta,\sigma}(z)=\mathbb E_{u\sim K_\sigma(\cdot-z)}Q_\theta(u)\). Latent-space mollification integrates the pole-type influence singularity instead of allowing one training sample near \(Q_\theta(z)\) to dominate the quantile feature or its gradient.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: The Influence Function of Transport-based Quantiles arXiv:2607.19080
Unverified 2026

Hard-Rod Symmetry Invariant Module

Construct a scalar feature or critic for oscillator-based neural dynamics that is invariant under the transformations imposed by free harmonic motion and elastic collisions. For finite-size rods, the module should represent only quantities compatible with common oscillator-phase rotations and momentum permutations, preventing a learned world model from inventing coordinate-dependent pseudo-conserved quantities that disappear after collisions.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Absence of hidden analytic conserved quantities in harmonically confined rods arXiv:2607.18872
✗ Mechanism failed 2026

Resolution-adaptive spectral front end

Replace a fixed Fourier or spectral resolution in a neural operator or sequence model with a data-adaptive spectral cutoff. Keep only modes whose estimated signal energy exceeds the noise-amplification and discretization floor implied by the available number of trajectories and samples per trajectory. This should reduce overfitting to high-frequency sensor noise and preserve accuracy when the same model is deployed at different sampling resolutions.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: The Cost of Discretization in Functional Linear Regression: Minimax Rates and Adaptation arXiv:2607.09350
Unverified 2026

Noise-crossing band-pass neuron

Replace selected ReLU or sigmoid units with a stochastic binary crossing activation that fires only when exactly one of two independent noise thresholds is crossed. The resulting expected activation is low for inputs far below or far above the noise distribution and maximal near its median, creating an analytically controlled band-pass and potentially reducing saturation-driven instability.

Useful5/10
Difficulty4/10
Novelty6/10
Paper: Reconstructing Backpropagation from Forward Fluctuations in Noise-modulated Neural Networks arXiv:2607.26483
Unverified 2026

Projective Compact Prior for Latent Models

Use the calibrated compact-support maximum-entropy law as a latent prior or representation regularizer in a VAE or autoencoder. Unlike a Gaussian prior, it prevents latent codes from drifting arbitrarily far while retaining explicitly controlled mean and covariance.

Useful5/10
Difficulty5/10
Novelty7/10
Paper: Projective Maximum Entropy: Universality and Acceptance-Region Calibration arXiv:2607.16547
Unverified 2026

Coherent-Fluctuating Amplitude Units

Represent selected hidden features as z = sqrt(N) exp(i theta), with a persistent phase and an explicitly stochastic amplitude. Regularize the ratio between coherent power |E[z]|^2 and total power E[|z|^2] toward the condensate prediction pi/4, while optionally matching higher amplitude moments.

Useful4/10
Difficulty5/10
Novelty8/10
Paper: Coherent Bose-Einstein condensation with fluctuating density arXiv:2607.12926