Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✗ Mechanism failed 2026

SU(d) Spectral Associative Memory

Replace vector-valued Hopfield neurons by SU(d)-valued latent states and construct Hebbian couplings from matrix memories. Recall is performed by iterating toward the dominant eigenmode of the induced lifted coupling operator, with each iterate projected back onto SU(d); the larger matrix representation should reduce random crosstalk and increase critical memory capacity.

Useful8/10
Difficulty7/10
Novelty8/10
Paper: High-Capacity Generalized Hopfield Networks arXiv:2608.08226
✗ Mechanism failed 2026

Front-Calibrated Bistable Neural Field

Construct a spatial recurrent network whose local vector hidden state has two stable attractors and whose neighbor coupling is diffusive. Train or constrain the network so that the desired attractor invades the undesired one with a controlled positive front velocity, rather than relying on a scalar class-frequency variable that can erase depletion and interface structure.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Nucleation beyond Equilibrium: Fronts Control Invasion in Bistable Ecosystems arXiv:2608.05251
✗ Mechanism failed 2026

Permutation-family residual network

Replace direct learning of a highly cancelling signed observable with a quotient-space model over symmetry orbits of inputs. Predict a physically constrained baseline for each family and use an LSTM or set/graph encoder only for the residual many-body correlation, then aggregate family predictions with known signed weights instead of forming a noisy sample-level ratio.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Learning the Fermion sign structure in path-integral Monte Carlo arXiv:2607.15060
✗ Mechanism failed 2026

Dendritic LMS State for Spiking ICL

Give a single spiking layer a persistent vector-valued apical compartment that stores the current online linear predictor for the task. On each labeled context pair, its subthreshold state performs a leaky LMS update; on the query, the state is read without updating, allowing in-context adaptation without attention or inference-time synaptic plasticity.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Dendritic In-Context Learning in a Single-Layer Spiking Neural Network arXiv:2607.02283
✗ Mechanism failed 2026

Fractional-memory recurrent state

Construct an efficient recurrent or state-space layer whose impulse response follows Mittag-Leffler relaxation instead of a single exponential. A bank of stable diagonal state channels approximates the long power-law tail, allowing the layer to retain information over widely separated timescales with only \(K\) states per feature.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Anomalous diffusion memory factorization: Characteristic timescales and application to inverse problem arXiv:2608.21674
✗ Mechanism failed 2026

Volume-Threshold Contracting State Layer

Construct a recurrent or state-space layer as a skew product: an expanding bounded feature coordinate drives a linearly contracting hidden state. Constrain the hidden transition matrix A to have spectral radius below one, and monitor the predicted transition ell times the absolute determinant of A equals one: below it, hidden trajectories should occupy a thin or fractal set, while above it they should have substantially higher-dimensional state coverage without losing local contraction.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Geometric Properties of Higher Dimensional Solenoidal Attractors arXiv:2607.27089
✗ Mechanism failed 2026

Autocatalytic Hysteresis Memory Cell

Replace or augment a recurrent hidden coordinate with a nonnegative bistable autocatalytic state driven by an external control signal. The cell retains information through metastable low and high states, while a periodic or slowly varying control produces a controlled phase lag and hysteresis useful for temporal regime detection. Explicit noise can be injected to test whether it enhances switching near the predicted intermediate-frequency regime.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Dynamic hysteresis in an autocatalytic reaction network arXiv:2607.24163
✗ Mechanism failed 2026

Critical Spectral Mode Compression

Replace a large diagonalizable recurrent or state-space transition operator by a sparse set of retained oscillatory modes selected according to their contribution to the output autocorrelation. Unlike magnitude-based pruning, the objective is to preserve the power-law return signal generated by pairwise spectral differences, enabling long memory with far fewer modes.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Information Compression at Criticality arXiv:2607.18388
✗ Mechanism failed 2026

Volterra BBP multi-timescale recurrent initialization

Initialize a recurrent or state-space transition matrix with weak Wigner noise plus a shared cumulative-sum correlation structure. Increasing the correlation strength produces recurrent eigenmodes one at a time at analytically predicted BBP thresholds, yielding a controllable hierarchy of short- and long-memory modes. The matrix should then be globally rescaled or constrained so that all active modes remain inside the desired stability radius.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: A Cascade of Volterra-Operator BBP Transitions in a Correlated Wigner Matrix arXiv:2607.10503
✗ Mechanism failed 2026

DMDc-Initialized Marginal-Stable Neural SSM

Replace the unconstrained transition of a recurrent or state-space neural network with a DMDc-initialized linear latent transition plus a learned nonlinear residual. Estimate the transition from a short warm-up dataset using Hankel delay coordinates, retain eigenmodes with decay rates near the unit circle for long-term memory, and let the neural residual model dynamics not explained by the linear backbone. This should make long-horizon prediction and slowly varying signals easier to learn while…

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Koopman Spectral Analysis of Lithium-Ion Battery Dynamics: State of Charge as a Marginally Stable Observable arXiv:2607.07594
✗ Mechanism failed 2026

Subresonant Power-Law Memory Bank

Add a deterministic complex-valued state-space bank whose mode detunings become progressively smaller with mode index, Delta_n=c n^{-p}, while input couplings decay as B_n=b n^{-kappa}. For slowly varying or constant forcing, the summed state follows the paper's subresonant response and grows like t^{1-alpha}, providing controllable power-law memory with only O(N) recurrent state updates. This should improve long-context retention compared with a same-size unconstrained RNN or uniformly spaced…

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Small Denominators and Subresonant Accumulation in Weakly Nonlinear Dispersive Dynamics arXiv:2607.01447
✗ Mechanism failed 2026

Canonical Tropical Segment State

Build a sequence or graph module whose state is a canonical set of affine tropical pieces rather than an opaque hidden vector. Compose consecutive segments by Minkowski addition of their lifted supports, merge alternatives by union followed by lower-hull reduction, and evaluate the resulting piecewise-linear function with a minimum.

Useful7/10
Difficulty7/10
Novelty8/10
Paper: On the Representational Geometry of Dynamic Programs arXiv:2608.25034
✗ Mechanism failed 2026

Delay-Ring Replay Memory

Replace a directed sequence-memory chain with a circular recurrent state propagated by a learned delayed convolution. The same learned kernel can support forward and reverse replay because replay direction is a dynamical mode of the ring, rather than requiring plasticity to explicitly learn both forward and backward synapses.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: Forward and reverse delay-driven hippocampal replay without symmetric plasticity arXiv:2608.21814
✗ Mechanism failed 2026

Thermodynamic Two-State Expert Gate

Add a slow latent two-state gate to a recurrent, state-space, or world-model network so that separate experts represent two qualitatively different dynamical regimes. Train the gate using the paper's two-state population and fluctuation mechanism rather than allowing an unconstrained softmax to average incompatible regimes. The model should allocate extra capacity near the gate's susceptibility peak, where regime uncertainty and forecast variance are predicted to be largest.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Structural Origin of Water Heat Capacity Anomaly from Classical and Quantum Simulations arXiv:2607.08957
✗ Mechanism failed 2026

Innovation-Compensated Latent Policy

In a partially observed reinforcement-learning or model-based control agent, expose the state-estimator innovation to the action head through a dedicated residual feedback branch. The policy produces a nominal action from the estimated latent state, while a learned innovation-compensation branch corrects actions when observations disagree with predicted latent dynamics. This explicitly separates nominal policy behavior from estimation-induced corrections and should help during fast transients…

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Revisiting Certainty Equivalence: The Structural Coupling Between Estimation and Control in Underactuated Nonlinear Systems arXiv:2607.07276
✗ Mechanism failed 2026

Periodic CMV Unitary Recurrent Layer

Replace a dense recurrent transition matrix with a periodic CMV-style product of alternating local 2x2 unitary cores. The transition is exactly norm-preserving, has O(n) trainable parameters under periodic tying, and can be applied through local factor operations rather than stored as an n-by-n matrix. Use turnover refactorization when changing the ordering or boundary connection of cores, enabling a compact cyclic unitary state-space layer.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Fast computation of eigenvalues of periodic CMV matrices arXiv:2607.06400
✗ Mechanism failed 2026

Heisenberg latent upsampler

Represent each latent state as a Heisenberg-group element and replace Euclidean interpolation in an upsampling or recurrent transition block by a four-point horizontal refinement plus the exact central signed-area correction. The module preserves the geometry of noncommutative composition, allowing the central latent coordinate to encode path-dependent information that ordinary coordinate-wise interpolation discards.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: A Heisenberg Subdivision Scheme with Central Smoothness Loss arXiv:2607.05446
✗ Mechanism failed 2026

Dual-unitary recurrent state block

Replace a generic recurrent transition with two coupled unitary transitions that share one block column and differ by a sign on the other block column. Each transition preserves hidden-state norm exactly, while the structured difference gives a controlled two-path recurrent architecture for long-context modeling.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Operator model and a trace formula for pairs of unitary operators arXiv:2607.05334