Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✗ Failed on benchmark 2026

Topology-Aware Streaming Jacobian Monitor

For a recurrent or graph neural network with known local connectivity, estimate each node's local Jacobian row using only graph neighbors rather than all hidden coordinates. Use the resulting sparse Jacobian both to compute a contraction certificate and to regularize training toward dynamically local interactions, reducing estimator variance and the number of samples required for reliable stability decisions.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Streaming Contraction Certificates for Nonlinear Networks: Topology-Aware Data Sufficiency with Partial Observation arXiv:2607.10893
△ Mechanism confirmed, baseline not beaten 2026

Critical Cross-Layer Weight Sharing

Construct deep or recurrent networks whose layer weights are correlated across depth with a prescribed power-law covariance, rather than either fully tying or fully independently sampling layers. The paper predicts two usable design boundaries: \(\gamma=1/2\) for divergence of correlation-induced fourth moments and \(\gamma=1\) for loss of summable-correlation flatness.

Useful7/10
Difficulty6/10
Novelty8/10
Paper: Bulk Phase Transition and Edge Behavior in Temporally Correlated Random Matrices arXiv:2608.23944
✗ Failed on benchmark 2026

Correlation-Exponent-Safe Weight Initialization

Initialize each row of a neural weight matrix as a stationary correlated Gaussian process instead of using independent entries, but constrain its correlation tail to remain on the finite-fourth-moment side of the transition. This creates controllable structured spectra while avoiding the heavy-edge regime predicted for correlations slower than \(t^{-1/2}\).

Useful7/10
Difficulty4/10
Novelty7/10
Paper: Bulk Phase Transition and Edge Behavior in Temporally Correlated Random Matrices arXiv:2608.23944
△ Mechanism confirmed, baseline not beaten 2026

Koopman Hankel Dual Autoencoder

Replace pointwise sequence reconstruction with reconstruction of overlapping past and future Hankel windows in a shared latent manifold. A first encoder compresses the delay-coordinate trajectory, while a second decoder or predictor reconstructs the future block from the latent state; training therefore penalizes representations that fit observations but do not preserve dynamical evolution.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Fault detection on manifolds of nonlinear dynamical systems with dual autoencoders arXiv:2608.17698
✗ Failed on benchmark 2026

Sensitivity-Conditioned Neural ODE Pruning

Use trajectory sensitivities to remove neural units or parameter groups whose effects are redundant over the available data support. A parameter group is pruned when its Fisher contribution is small or its sensitivity is nearly collinear with other groups, producing a compact neural ODE without relying only on parameter magnitude.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: Identifiability-aware neural ordinary differential equations for parsimonious and reliable dynamic modelling arXiv:2608.13044
△ Mechanism confirmed, baseline not beaten 2026

Noise-Triggered Latent Rank Adaptation

Use the recursive errors-in-variables subspace spectrum as a controller for the width of a latent SSM rather than fixing the state dimension in advance. Neurons or state channels are added when corrected covariance eigenvalues rise above the noise floor and pruned when they remain below it, producing a model-order-adaptive recurrent architecture for nonstationary streams.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: A recursive subspace based method for errors-in-variables model identification of time-varying systems arXiv:2607.17065
✓✓ Beats tuned baseline 2026

Multiplicative Manifold Unscented Recurrent Cell

Replace the Euclidean hidden-state update of a recurrent or state-space neural network with a mixed manifold state containing a rotation component and Euclidean features. Propagate uncertainty with sigma points in tangent error coordinates, retract rotational perturbations through the exponential map, and compute the training loss from the predicted covariance. This avoids invalid rotations and captures second-order curvature effects that a first-order EKF-style recurrent cell misses at large…

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Fully Multiplicative Attitude and Orbit Determination for Deep space Navigation arXiv:2607.10072
△ Mechanism confirmed, baseline not beaten 2026

Mittag-Leffler Memory Diagnostic and Gating

Measure how validation forecast error grows with prediction horizon and fit exponential and Mittag-Leffler models. When the Mittag-Leffler fit is decisively better, activate a fractional-memory SSM or long-memory residual branch and use its fitted effective order to set the branch's kernel decay and horizon-loss weights; otherwise retain a conventional recurrent or finite-memory branch.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Mittag-Leffler-Type Forecast-Error Growth as a Diagnostic Indicator of Fractional Dynamics arXiv:2607.08588
Unverified 2026

PCA-Discovered Implicit Latent Dynamics

Replace an explicit recurrent transition with a learned descriptor relation in latent space, allowing some latent coordinates to satisfy algebraic constraints rather than being numerically integrated. Fit the relation using total-least-squares or iterative PCA on the jointly observed trajectory, so noise in every channel is treated symmetrically and the model can discover whether the latent system is index-0 or index-1.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Subspace Based Identification of Errors-in-Variables Linear Descriptor Systems arXiv:2608.30259
Unverified 2026

Data-Identified Neural Dependency Graph

Partition a neural state or feature vector into blocks and identify directed dependencies between blocks from one-step transition data. Use the inferred design structure matrix as a hard mask or soft gate on recurrent, state-space, graph, or mixture-of-experts couplings, replacing a dense unconstrained interaction matrix with a data-supported sparse graph.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Novel methodology for obtaining design structure matrices using network identification arXiv:2608.16759
Unverified 2026

Measure-Lifted Entropy Amplifier

Replace a deterministic latent state with a probability measure over latent states, represented by particles or weighted prototypes. Apply the learned latent transition to every particle, so one base trajectory map induces a dynamics on distributions; use an entropy-preservation or entropy-growth regularizer to prevent collapse of the ensemble. The mechanism predicts that any positive base-state trajectory entropy can generate unbounded distinguishability in the ideal measure space through…

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Entropies of compact subsets and supported measures arXiv:2608.09702
Unverified 2026

Log-Rank Fluctuation Channels

Augment a causal sequence model with a small hierarchy of prefix summaries weighted by powers of the logarithmic rank of each preceding token. The summaries retain order-sensitive deviations from a baseline representation while costing O(KNd) for sequence length N, hierarchy width K, and hidden dimension d, instead of O(N^2d) dense attention.

Useful6/10
Difficulty4/10
Novelty6/10
Paper: A Logarithmic Fluctuation Hierarchy for Sequential Interacting Diffusions arXiv:2607.22470
Unverified 2026

Hard-Rod Symmetry Invariant Module

Construct a scalar feature or critic for oscillator-based neural dynamics that is invariant under the transformations imposed by free harmonic motion and elastic collisions. For finite-size rods, the module should represent only quantities compatible with common oscillator-phase rotations and momentum permutations, preventing a learned world model from inventing coordinate-dependent pseudo-conserved quantities that disappear after collisions.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Absence of hidden analytic conserved quantities in harmonically confined rods arXiv:2607.18872
✗ Mechanism failed 2026

Resolution-adaptive spectral front end

Replace a fixed Fourier or spectral resolution in a neural operator or sequence model with a data-adaptive spectral cutoff. Keep only modes whose estimated signal energy exceeds the noise-amplification and discretization floor implied by the available number of trajectories and samples per trajectory. This should reduce overfitting to high-frequency sensor noise and preserve accuracy when the same model is deployed at different sampling resolutions.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: The Cost of Discretization in Functional Linear Regression: Minimax Rates and Adaptation arXiv:2607.09350
✓✓ Beats tuned baseline 2026

Private spectral whitening front-end

Estimate the temporal spectrum of each sequence channel using a locally private procedure, then apply a regularized inverse-square-root spectral filter before the sequence enters attention or an SSM. The filter removes predictable low-frequency or narrow-band redundancy while avoiding unstable amplification at frequencies where the private estimate is small.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: On the privacy cost for dependent Gaussian data: spectral density estimation under local differential privacy arXiv:2608.24847
Unverified 2026

Correlation-Length Scaling Benchmark

Treat the maximum dependency distance faithfully modeled by a finite neural architecture as an emergent correlation length, and estimate how it grows with depth, state size, or attention span. Fit the exponent \(\kappa\) and use it as an architecture-selection signal: a model with larger \(\kappa\) should acquire long-range competence more efficiently at equal parameter or FLOP budget.

Useful5/10
Difficulty3/10
Novelty6/10
Paper: Universal scaling framework for parameterized quantum evolutions at criticality arXiv:2607.22863
Unverified 2026

SURE-Adaptive Derivative Front End

Prepend an adaptive Savitzky-Golay derivative bank to a temporal neural network. For each input channel and derivative order, select the local window by minimizing Stein's unbiased risk estimate, then concatenate the raw signal with the estimated derivatives. This supplies denoised velocity and acceleration features without requiring clean derivative targets or forcing the backbone to learn unstable finite-difference filters.

Useful5/10
Difficulty3/10
Novelty6/10
Paper: PDE Identification Using Noise Adaptive Differentiation in Strong Form (S-IDENT) arXiv:2606.31776