Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✗ Mechanism failed 2026

Permutation-family residual network

Replace direct learning of a highly cancelling signed observable with a quotient-space model over symmetry orbits of inputs. Predict a physically constrained baseline for each family and use an LSTM or set/graph encoder only for the residual many-body correlation, then aggregate family predictions with known signed weights instead of forming a noisy sample-level ratio.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Learning the Fermion sign structure in path-integral Monte Carlo arXiv:2607.15060
✗ Failed on benchmark 2026

Topology-Aware Streaming Jacobian Monitor

For a recurrent or graph neural network with known local connectivity, estimate each node's local Jacobian row using only graph neighbors rather than all hidden coordinates. Use the resulting sparse Jacobian both to compute a contraction certificate and to regularize training toward dynamically local interactions, reducing estimator variance and the number of samples required for reliable stability decisions.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Streaming Contraction Certificates for Nonlinear Networks: Topology-Aware Data Sufficiency with Partial Observation arXiv:2607.10893
△ Mechanism confirmed, baseline not beaten 2026

Critical Cross-Layer Weight Sharing

Construct deep or recurrent networks whose layer weights are correlated across depth with a prescribed power-law covariance, rather than either fully tying or fully independently sampling layers. The paper predicts two usable design boundaries: \(\gamma=1/2\) for divergence of correlation-induced fourth moments and \(\gamma=1\) for loss of summable-correlation flatness.

Useful7/10
Difficulty6/10
Novelty8/10
Paper: Bulk Phase Transition and Edge Behavior in Temporally Correlated Random Matrices arXiv:2608.23944
△ Mechanism confirmed, baseline not beaten 2026

Koopman Hankel Dual Autoencoder

Replace pointwise sequence reconstruction with reconstruction of overlapping past and future Hankel windows in a shared latent manifold. A first encoder compresses the delay-coordinate trajectory, while a second decoder or predictor reconstructs the future block from the latent state; training therefore penalizes representations that fit observations but do not preserve dynamical evolution.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Fault detection on manifolds of nonlinear dynamical systems with dual autoencoders arXiv:2608.17698
✗ Failed on benchmark 2026

Sensitivity-Conditioned Neural ODE Pruning

Use trajectory sensitivities to remove neural units or parameter groups whose effects are redundant over the available data support. A parameter group is pruned when its Fisher contribution is small or its sensitivity is nearly collinear with other groups, producing a compact neural ODE without relying only on parameter magnitude.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: Identifiability-aware neural ordinary differential equations for parsimonious and reliable dynamic modelling arXiv:2608.13044
✓✓ Beats tuned baseline 2026

Multiplicative Manifold Unscented Recurrent Cell

Replace the Euclidean hidden-state update of a recurrent or state-space neural network with a mixed manifold state containing a rotation component and Euclidean features. Propagate uncertainty with sigma points in tangent error coordinates, retract rotational perturbations through the exponential map, and compute the training loss from the predicted covariance. This avoids invalid rotations and captures second-order curvature effects that a first-order EKF-style recurrent cell misses at large…

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Fully Multiplicative Attitude and Orbit Determination for Deep space Navigation arXiv:2607.10072
△ Mechanism confirmed, baseline not beaten 2026

Mittag-Leffler Memory Diagnostic and Gating

Measure how validation forecast error grows with prediction horizon and fit exponential and Mittag-Leffler models. When the Mittag-Leffler fit is decisively better, activate a fractional-memory SSM or long-memory residual branch and use its fitted effective order to set the branch's kernel decay and horizon-loss weights; otherwise retain a conventional recurrent or finite-memory branch.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Mittag-Leffler-Type Forecast-Error Growth as a Diagnostic Indicator of Fractional Dynamics arXiv:2607.08588
△ Mechanism confirmed, baseline not beaten 2026

Wasserstein-Budgeted Width Allocation

Use the paper's finite-width O(n^{-1/2}) Gaussian-process approximation bound as a width-budgeting rule rather than choosing every hidden dimension uniformly. Estimate an architecture-specific constant for each layer or attention contraction, then allocate width according to the smallest dimension satisfying its allowed distributional error. This should produce narrower models at comparable GP-like behavior, or permit the same parameter budget to be concentrated in the layers where finite-width…

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Quantitative Gaussian-Process limits of Tensor Programs arXiv:2607.06290
Unverified 2026

PCA-Discovered Implicit Latent Dynamics

Replace an explicit recurrent transition with a learned descriptor relation in latent space, allowing some latent coordinates to satisfy algebraic constraints rather than being numerically integrated. Fit the relation using total-least-squares or iterative PCA on the jointly observed trajectory, so noise in every channel is treated symmetrically and the model can discover whether the latent system is index-0 or index-1.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Subspace Based Identification of Errors-in-Variables Linear Descriptor Systems arXiv:2608.30259
Unverified 2026

Data-Identified Neural Dependency Graph

Partition a neural state or feature vector into blocks and identify directed dependencies between blocks from one-step transition data. Use the inferred design structure matrix as a hard mask or soft gate on recurrent, state-space, graph, or mixture-of-experts couplings, replacing a dense unconstrained interaction matrix with a data-supported sparse graph.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Novel methodology for obtaining design structure matrices using network identification arXiv:2608.16759
Unverified 2026

Measure-Lifted Entropy Amplifier

Replace a deterministic latent state with a probability measure over latent states, represented by particles or weighted prototypes. Apply the learned latent transition to every particle, so one base trajectory map induces a dynamics on distributions; use an entropy-preservation or entropy-growth regularizer to prevent collapse of the ensemble. The mechanism predicts that any positive base-state trajectory entropy can generate unbounded distinguishability in the ideal measure space through…

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Entropies of compact subsets and supported measures arXiv:2608.09702
Unverified 2026

Hard-Rod Symmetry Invariant Module

Construct a scalar feature or critic for oscillator-based neural dynamics that is invariant under the transformations imposed by free harmonic motion and elastic collisions. For finite-size rods, the module should represent only quantities compatible with common oscillator-phase rotations and momentum permutations, preventing a learned world model from inventing coordinate-dependent pseudo-conserved quantities that disappear after collisions.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Absence of hidden analytic conserved quantities in harmonically confined rods arXiv:2607.18872
Unverified 2026

SURE-Adaptive Derivative Front End

Prepend an adaptive Savitzky-Golay derivative bank to a temporal neural network. For each input channel and derivative order, select the local window by minimizing Stein's unbiased risk estimate, then concatenate the raw signal with the estimated derivatives. This supplies denoised velocity and acceleration features without requiring clean derivative targets or forcing the backbone to learn unstable finite-difference filters.

Useful5/10
Difficulty3/10
Novelty6/10
Paper: PDE Identification Using Noise Adaptive Differentiation in Strong Form (S-IDENT) arXiv:2606.31776
Unverified 2026

Coherent-Fluctuating Amplitude Units

Represent selected hidden features as z = sqrt(N) exp(i theta), with a persistent phase and an explicitly stochastic amplitude. Regularize the ratio between coherent power |E[z]|^2 and total power E[|z|^2] toward the condensate prediction pi/4, while optionally matching higher amplitude moments.

Useful4/10
Difficulty5/10
Novelty8/10
Paper: Coherent Bose-Einstein condensation with fluctuating density arXiv:2607.12926