Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✗ Mechanism failed 2026

SU(d) Spectral Associative Memory

Replace vector-valued Hopfield neurons by SU(d)-valued latent states and construct Hebbian couplings from matrix memories. Recall is performed by iterating toward the dominant eigenmode of the induced lifted coupling operator, with each iterate projected back onto SU(d); the larger matrix representation should reduce random crosstalk and increase critical memory capacity.

Useful8/10
Difficulty7/10
Novelty8/10
Paper: High-Capacity Generalized Hopfield Networks arXiv:2608.08226
✗ Mechanism failed 2026

Front-Calibrated Bistable Neural Field

Construct a spatial recurrent network whose local vector hidden state has two stable attractors and whose neighbor coupling is diffusive. Train or constrain the network so that the desired attractor invades the undesired one with a controlled positive front velocity, rather than relying on a scalar class-frequency variable that can erase depletion and interface structure.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Nucleation beyond Equilibrium: Fronts Control Invasion in Bistable Ecosystems arXiv:2608.05251
✗ Mechanism failed 2026

Lie-Bracket Fourier Mixer

Construct a mode-mixing layer from a few sphere-preserving vector fields and shift operators rather than a dense learned Fourier convolution. A commutator of two low-complexity shifted rotations produces a new interaction at the sum of their offsets, allowing long-range or high-frequency mode coupling to be synthesized with only a small number of primitive operators. The layer can be used whenever each feature vector is normalized to the sphere, or more generally as a norm-preserving block on…

Useful8/10
Difficulty5/10
Novelty8/10
Paper: Sphere Constraints and Harmonic Map Flow: Controllability and Reachability by Low-Mode Forcing arXiv:2607.05687
✗ Mechanism failed 2026

Dendritic LMS State for Spiking ICL

Give a single spiking layer a persistent vector-valued apical compartment that stores the current online linear predictor for the task. On each labeled context pair, its subthreshold state performs a leaky LMS update; on the query, the state is read without updating, allowing in-context adaptation without attention or inference-time synaptic plasticity.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Dendritic In-Context Learning in a Single-Layer Spiking Neural Network arXiv:2607.02283
✗ Mechanism failed 2026

Correlated Long-Range Residual Mixer

Replace an iid local or randomly sparse residual mixer with a distance-decaying long-range operator whose edge magnitudes are correlated through a shared latent Gaussian field. The paper predicts that these correlations qualitatively change low-energy spectral scaling and increase multiscale information propagation relative to iid long-range weights. Apply the operator as a spectrally normalized residual block so that the benefit comes from correlated scale coverage rather than uncontrolled…

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Effective Hamiltonian description on monitored Majorana chains: correlated power-law hoppings and unconventional entanglement scaling arXiv:2609.04091
✗ Mechanism failed 2026

Fractional-memory recurrent state

Construct an efficient recurrent or state-space layer whose impulse response follows Mittag-Leffler relaxation instead of a single exponential. A bank of stable diagonal state channels approximates the long power-law tail, allowing the layer to retain information over widely separated timescales with only \(K\) states per feature.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Anomalous diffusion memory factorization: Characteristic timescales and application to inverse problem arXiv:2608.21674
✗ Mechanism failed 2026

Volume-Threshold Contracting State Layer

Construct a recurrent or state-space layer as a skew product: an expanding bounded feature coordinate drives a linearly contracting hidden state. Constrain the hidden transition matrix A to have spectral radius below one, and monitor the predicted transition ell times the absolute determinant of A equals one: below it, hidden trajectories should occupy a thin or fractal set, while above it they should have substantially higher-dimensional state coverage without losing local contraction.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Geometric Properties of Higher Dimensional Solenoidal Attractors arXiv:2607.27089
✗ Mechanism failed 2026

Autocatalytic Hysteresis Memory Cell

Replace or augment a recurrent hidden coordinate with a nonnegative bistable autocatalytic state driven by an external control signal. The cell retains information through metastable low and high states, while a periodic or slowly varying control produces a controlled phase lag and hysteresis useful for temporal regime detection. Explicit noise can be injected to test whether it enhances switching near the predicted intermediate-frequency regime.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Dynamic hysteresis in an autocatalytic reaction network arXiv:2607.24163
✗ Mechanism failed 2026

Dyadic Resolvent Filter Layer

Replace a single polynomial graph filter or dense inverse with a multiscale layer that applies unitary propagations at dyadic times and combines them according to the dyadic resolvent identity. For a symmetric graph operator, this implements a stable rational spectral filter that can selectively retain or suppress frequency bands while exposing logarithmic multiscale structure.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Dyadic Resolvent Representations of Self-Adjoint Operators: Propagator Expansions, Spectral Measures, and Zeta Functions arXiv:2607.21278
✗ Mechanism failed 2026

Critical Spectral Mode Compression

Replace a large diagonalizable recurrent or state-space transition operator by a sparse set of retained oscillatory modes selected according to their contribution to the output autocorrelation. Unlike magnitude-based pruning, the objective is to preserve the power-law return signal generated by pairwise spectral differences, enabling long memory with far fewer modes.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Information Compression at Criticality arXiv:2607.18388
✗ Mechanism failed 2026

Memory-Retaining RG Feature Blocks

Replace scale-blind pooling or downsampling with a coarse-graining block that carries an explicit relevant scale variable \(\eta\) alongside the feature field. The block is constrained to represent features in the memory-retaining form \(h(\xi,\eta)=\eta^{\alpha}F(\xi/\eta^{\beta})\), allowing both feature amplitude and profile shape to depend on the scale inherited from the input or previous RG step.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Memory Retention and the Classification of Renormalization-Group Fixed Points in Self-Similar Dynamics arXiv:2607.14388
✗ Mechanism failed 2026

Volterra BBP multi-timescale recurrent initialization

Initialize a recurrent or state-space transition matrix with weak Wigner noise plus a shared cumulative-sum correlation structure. Increasing the correlation strength produces recurrent eigenmodes one at a time at analytically predicted BBP thresholds, yielding a controllable hierarchy of short- and long-memory modes. The matrix should then be globally rescaled or constrained so that all active modes remain inside the desired stability radius.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: A Cascade of Volterra-Operator BBP Transitions in a Correlated Wigner Matrix arXiv:2607.10503
✗ Mechanism failed 2026

DMDc-Initialized Marginal-Stable Neural SSM

Replace the unconstrained transition of a recurrent or state-space neural network with a DMDc-initialized linear latent transition plus a learned nonlinear residual. Estimate the transition from a short warm-up dataset using Hankel delay coordinates, retain eigenmodes with decay rates near the unit circle for long-term memory, and let the neural residual model dynamics not explained by the linear backbone. This should make long-horizon prediction and slowly varying signals easier to learn while…

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Koopman Spectral Analysis of Lithium-Ion Battery Dynamics: State of Charge as a Marginally Stable Observable arXiv:2607.07594
✗ Mechanism failed 2026

Subresonant Power-Law Memory Bank

Add a deterministic complex-valued state-space bank whose mode detunings become progressively smaller with mode index, Delta_n=c n^{-p}, while input couplings decay as B_n=b n^{-kappa}. For slowly varying or constant forcing, the summed state follows the paper's subresonant response and grows like t^{1-alpha}, providing controllable power-law memory with only O(N) recurrent state updates. This should improve long-context retention compared with a same-size unconstrained RNN or uniformly spaced…

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Small Denominators and Subresonant Accumulation in Weakly Nonlinear Dispersive Dynamics arXiv:2607.01447
✗ Mechanism failed 2026

Delay-Ring Replay Memory

Replace a directed sequence-memory chain with a circular recurrent state propagated by a learned delayed convolution. The same learned kernel can support forward and reverse replay because replay direction is a dynamical mode of the ring, rather than requiring plasticity to explicitly learn both forward and backward synapses.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: Forward and reverse delay-driven hippocampal replay without symmetric plasticity arXiv:2608.21814
✗ Mechanism failed 2026

Algebraic Pinch-Curve Spectral Layer

Replace a dense learnable Fourier multiplier with a low-parameter multiplier concentrated near the common zero set of two polynomial constraint symbols. A linear constraint together with a cubic constraint can produce straight or curved frequency loci, allowing the network to represent directional long-range structure while using far fewer spectral parameters than a full 3D frequency grid.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Symmetry-Protected Pinch Curves in Classical Spin Liquids arXiv:2607.09470
✗ Mechanism failed 2026

Resolution-adaptive spectral front end

Replace a fixed Fourier or spectral resolution in a neural operator or sequence model with a data-adaptive spectral cutoff. Keep only modes whose estimated signal energy exceeds the noise-amplification and discretization floor implied by the available number of trajectories and samples per trajectory. This should reduce overfitting to high-frequency sensor noise and preserve accuracy when the same model is deployed at different sampling resolutions.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: The Cost of Discretization in Functional Linear Regression: Minimax Rates and Adaptation arXiv:2607.09350
✗ Mechanism failed 2026

Thermodynamic Two-State Expert Gate

Add a slow latent two-state gate to a recurrent, state-space, or world-model network so that separate experts represent two qualitatively different dynamical regimes. Train the gate using the paper's two-state population and fluctuation mechanism rather than allowing an unconstrained softmax to average incompatible regimes. The model should allocate extra capacity near the gate's susceptibility peak, where regime uncertainty and forecast variance are predicted to be largest.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Structural Origin of Water Heat Capacity Anomaly from Classical and Quantum Simulations arXiv:2607.08957
✗ Mechanism failed 2026

Periodic CMV Unitary Recurrent Layer

Replace a dense recurrent transition matrix with a periodic CMV-style product of alternating local 2x2 unitary cores. The transition is exactly norm-preserving, has O(n) trainable parameters under periodic tying, and can be applied through local factor operations rather than stored as an n-by-n matrix. Use turnover refactorization when changing the ordering or boundary connection of cores, enabling a compact cyclic unitary state-space layer.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Fast computation of eigenvalues of periodic CMV matrices arXiv:2607.06400
✗ Mechanism failed 2026

Heisenberg latent upsampler

Represent each latent state as a Heisenberg-group element and replace Euclidean interpolation in an upsampling or recurrent transition block by a four-point horizontal refinement plus the exact central signed-area correction. The module preserves the geometry of noncommutative composition, allowing the central latent coordinate to encode path-dependent information that ordinary coordinate-wise interpolation discards.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: A Heisenberg Subdivision Scheme with Central Smoothness Loss arXiv:2607.05446
✗ Mechanism failed 2026

Dual-unitary recurrent state block

Replace a generic recurrent transition with two coupled unitary transitions that share one block column and differ by a sign on the other block column. Each transition preserves hidden-state norm exactly, while the structured difference gives a controlled two-path recurrent architecture for long-context modeling.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Operator model and a trace formula for pairs of unitary operators arXiv:2607.05334
✗ Mechanism failed 2026

Pole-Certified SSM Initialization

Extract a small set of stable exponential modes from an observed neural sequence and use them to initialize a diagonal or block-diagonal state-space model. Hankel-pencil eigenvalues propose the modes, while persistence across shifts and contour margins reject modes caused by noise or a short-lived background.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Determinant Characteristics and Argument-Principle Certification for Visible Poles in Meromorphic Continuation arXiv:2607.04568
✗ Mechanism failed 2026

Positive Mellin Mixture Gate

Replace a free-form order-dependent gate with a positive mixture of Mellin powers $(1+s)^{-a}$. This gives a small, interpretable module whose response across the order variable is automatically generated by a positive measure and therefore inherits complete monotonicity, log-convexity, and Hankel-moment structure.

Useful6/10
Difficulty4/10
Novelty8/10
Paper: Order-Moment Transport and Hankel Determinants in Special-Function Inequalities arXiv:2606.31647