Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

Unverified 2026

Resonance-Gated Triadic Fourier Layer

Replace unconstrained spectral mixing with a three-component triadic interaction whose strength is determined by the quadratic phase mismatch R(xi,xi_1). Near-resonant products receive high weight because their phases remain coherent, while strongly nonresonant products are attenuated. The resonance bandwidth can be fixed from the frequency grid or learned as a positive parameter.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: On solitary wave solutions with two-frequency parameters to the three-component system of quadratic nonlinear Schrödinger equations arXiv:2608.12983
Unverified 2026

Intermittent Multi-Mode Memory Gate

Add a bounded routing state to an RNN, state-space model, or mixture-of-experts layer, with several neutral fixed points representing persistent modes. The state moves between modes when far from a fixed point but escapes each mode only polynomially when close to it, creating controllable long memory without setting a linear eigenvalue arbitrarily close to one. A temperature parameter selects between an entropy-rich phase using many modes and a low-entropy phase concentrated near one preferred…

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Thermodynamic formalism for intermittent maps with multiple neutral fixed points and phase transitions arXiv:2608.12784
Unverified 2026

Equivariant Critical-Mode Branch Seeding

When a symmetry-frequency block becomes critical, initialize or perturb the network specifically along its critical representation rather than injecting isotropic noise into all hidden channels. This creates trainable branches for the symmetry patterns predicted by the bifurcation calculation and can expose useful periodic solutions that ordinary symmetry-preserving training fails to reach.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Local and Global Equivariant Bifurcation for Periodic Weyl and Riesz Fractional Equations arXiv:2608.11101
Unverified 2026

Unit-circle-root Toeplitz mixer

Replace a freely learned finite impulse-response mixing kernel with a matrix polynomial whose roots are constrained to the unit circle. The resulting block-Toeplitz operator has an explicitly positive semidefinite spectral construction, while increasing the polynomial degree gives a systematic capacity knob for approximating matrix-valued frequency responses.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Pure matrix states on block Toeplitz matrices arXiv:2608.10701
Unverified 2026

Measure-Lifted Entropy Amplifier

Replace a deterministic latent state with a probability measure over latent states, represented by particles or weighted prototypes. Apply the learned latent transition to every particle, so one base trajectory map induces a dynamics on distributions; use an entropy-preservation or entropy-growth regularizer to prevent collapse of the ensemble. The mechanism predicts that any positive base-state trajectory entropy can generate unbounded distinguishability in the ideal measure space through…

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Entropies of compact subsets and supported measures arXiv:2608.09702
Unverified 2026

Scalene Nilpotent-Symmetry Network

Augment a sequence network with a learned staggered matrix-product-operator symmetry and penalize its commutator with the network map. Unlike ordinary equivariance, the auxiliary operator need not define a self-commuting transfer-matrix family: it can be discovered through cross-commutation with a second alternating operator, while nilpotency supplies a finite hierarchy of symmetry constraints. The model should preserve generalized symmetry sectors and exhibit lower commutator error on…

Useful6/10
Difficulty7/10
Novelty8/10
Paper: Scalene Yang--Baxter triples as a source of hidden symmetries beyond the ordinary Yang--Baxter equation arXiv:2608.09081
Unverified 2026

Single-Node Observable Leaky-RNN

Construct a sparse recurrent network with positive edge weights and Leaky-ReLU updates so that one selected hidden node, observed over a finite time window, contains enough information to reconstruct the full hidden state. Add an auxiliary decoder from the observed trajectory to the initial state or current state, and use graph rewiring or edge-growth until every hidden node has a directed path to the sensor within the observation horizon.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: On the Observability and Controllability of Leaky-ReLU Networks arXiv:2608.09059
Unverified 2026

Nielsen Quaternionic Hyperbolic Latent Layer

Represent each recurrent latent state as a pair of unit quaternions \((q_1,q_2)\in\mathrm{SU}(2)^2\), and evolve it with a composition of elementary Nielsen maps corresponding to a chosen hyperbolic matrix \(A\in\mathrm{SL}(2,\mathbb{Z})\). The layer exactly preserves the group manifold and Haar volume, preserves the commuting locus \(q_1q_2=q_2q_1\), and reproduces toral hyperbolic dynamics there, giving a structured long-horizon prior instead of an unconstrained matrix recurrence.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Quaternionic Extensions of Hyperbolic Toral Automorphisms arXiv:2608.08252
Unverified 2026

Parallel Phase Oscillator SSM

Replace real diagonal state-space channels with complex damped oscillators whose hidden states encode both amplitude and phase. Train with parallel causal convolution and deploy with the equivalent one-step recurrence, allowing the same layer to support efficient batched training and low-memory streaming inference.

Useful6/10
Difficulty5/10
Novelty4/10
Paper: Phase State Space Models: Parallel, Surrogate-Free Training of Spiking Networks arXiv:2608.07754
Unverified 2026

Wasserstein-Gated Cached Equilibrium Adapter

Add a small constrained equilibrium layer whose response depends on the current neural state and recent exogenous history, then cache responses keyed by a learned history embedding. For a new history, reuse a cached response only when an empirical Wasserstein distance to the cached history is below a threshold; otherwise run a few inner optimization iterations. The paper's local Holder and trajectory-stability results motivate graceful degradation rather than catastrophic errors for nearby…

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Stability of Differential Stochastic Variational Inequalities with History-Dependent Responses and Transfer Learning arXiv:2608.06923
Unverified 2026

KPZ Directed-Polymer Attention

Replace independent Gaussian attention noise or unconstrained token routing with a directed-polymer path distribution over positions and layers. The router aggregates exponentially many monotone paths through temporally correlated random edge scores, producing heavy-tailed but spatially coherent routing and preventing attention from collapsing onto a single token. The paper's t^{2/3} wandering and t^{1/3} free-energy fluctuations become measurable diagnostics and tunable targets rather than…

Useful6/10
Difficulty5/10
Novelty7/10
Paper: KPZ Superdiffusion of Local Correlators in Diffusive Random Quantum Circuits arXiv:2608.06459
Unverified 2026

Invertible Fourier Surrogate for Periodic Sequence Modeling

Represent periodic input-output behavior using a compact real vector of Fourier coefficients and learn an invertible neural map from input coefficients to output coefficients. Inference then obtains the input representation for a desired periodic output by a single inverse pass instead of iterative optimization through a nonlinear forward model, while the Fourier representation reduces sequence dimensionality when high-rate signals are spectrally sparse.

Useful6/10
Difficulty6/10
Novelty4/10
Paper: Certified Feedforward Tracking for Unknown Nonlinear Systems via Invertible Neural Networks arXiv:2608.06419
Unverified 2026

Power-Law Volterra Memory

Add a causal memory branch whose weights are generated by the paper's power-type Volterra kernel rather than learned independently at every lag. Learn or softly constrain the exponents so the model can select rough short-memory behavior or smoother long-memory behavior while using only a few parameters. The branch can be implemented as a truncated causal convolution, a multiresolution approximation, or a recurrent state-space realization.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Small ball probabilities and Chung's law of the iterated logarithm for Gaussian Volterra processes with power-type kernels arXiv:2608.05679
Unverified 2026

Contour-reduced parametric SSM

Replace repeated full-dimensional matrix-exponential or ODE solves in a conditioned continuous-time state-space layer with contour quadrature evaluated in a projection basis. The same reduced basis and contour nodes can serve many conditioning vectors, while shifted reduced resolvents provide a stable and differentiable approximation over a prescribed time window.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Contour integral methods and model order reduction for parametric linear control systems arXiv:2608.05363
Unverified 2026

KAM-Stabilized Quasiperiodic Recurrent Memory

Construct a recurrent module with a phase variable and a transverse memory coordinate modeled on a perturbed twist map. Train the transverse state to lie on an invariant graph over the phase, while the phase follows an approximately irrational rigid rotation. A KAM-inspired graph correction and residual penalty should reduce long-horizon drift in recurrent prediction.

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Persistence of invariant graphs for twist maps under analytic perturbations arXiv:2608.05239
Unverified 2026

Orientation-doubling positional channel

For local structures with a forward/reverse ambiguity, expose both ordered directions and add one explicit orientation bit. This creates a shared bidirectional positional encoder that can distinguish reflected neighborhoods without maintaining two completely independent directional encoders.

Useful6/10
Difficulty4/10
Novelty8/10
Paper: Information and Locality in Cayley Graphs arXiv:2608.04608
Unverified 2026

Curved-Critical Residual Lattice

Construct a 2D recurrent or residual neural lattice with slowly varying local couplings, while parameterizing those couplings so that an anisotropy invariant remains constant across all spatial and depth locations. The network obtains controlled local propagation velocities rather than arbitrary inhomogeneous amplification, enabling depth-dependent receptive fields while preserving near-critical signal propagation.

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Two dimensional inhomogeneous classical systems at criticality arXiv:2608.02903
Unverified 2026

Noncommutative controllability regularizer

Equip a recurrent or state-space layer with multiple noncommuting transition operators and regularize the span of finite operator words applied to the input injection matrix. This discourages hidden directions that cannot be reached from the input and may improve long-range input influence, gradient propagation, and robustness under operator switching.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: A noncommunicative Kalman condition for null controllability of backward stochastic parabolic systems arXiv:2608.01836
Unverified 2026

Schur-Agler contractive gated layer

Replace an unconstrained recurrent or residual transition with a block-structured contractive realization whose input-dependent multiplier is a direct sum of bounded branches. The resulting layer has a built-in non-expansive energy bound on the chosen normalized input domain, while still allowing different branches to respond to different coordinates or feature groups.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Function theory of the hexablock and applications to the tetrablock and Euclidean biball arXiv:2608.00819
Unverified 2026

Positive Bidiagonal Diffusion Layer

Replace a learned nonnegative banded transition matrix by an ordered product of learnable stochastic bidiagonal factors. Each factor performs one local left or right transport step, so a product of p lower and q upper factors creates an effective bandwidth of p+q while retaining nonnegative entries, row sums equal to one, and a highly structured propagation kernel.

Useful6/10
Difficulty3/10
Novelty7/10
Paper: Positive Bidiagonal Factorizations for Banded Markov Processes arXiv:2608.00788
Unverified 2026

Invariant-Preserving Latent Compression

Replace unconstrained low-rank compression of a neural state with an augmented basis that always contains vectors representing known conserved quantities or diagnostically important linear statistics. After each learned transition, project the state back onto the affine constraint set with an exact minimum-norm correction, preventing rank truncation and model error from accumulating in those statistics.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Structure-Preserving Dynamical Low-Rank Approximations for Stochastic Vlasov--Poisson Equations arXiv:2608.00397
Unverified 2026

Finite-Band Polynomial Seed Bank

Construct one reference Lanczos basis for a symmetric propagation operator H, then derive several seed-specific spectral responses for Q_a(H)x_0 through finite-band polynomial connectors. With degree-r seeds, each transformed basis vector uses at most 2r+1 neighboring reference basis vectors, avoiding a separate Lanczos factorization for every seed.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Polynomial Initial-State Jumps and Christoffel Transforms in Krylov Complexity arXiv:2607.05294
Unverified 2026

Pullback-Commuting 3-Axis Network

Use three learned state-transition operators corresponding to three data axes, and train them to satisfy the paper's pullback-style interchange rule. For every local pair of axes, two successive updates should reach the same square state; for triples of axes, all six update orders should agree. This reduces sensitivity to scan direction and limits long-horizon drift caused by inconsistent local transitions.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Higher-Dimensional Symbolic Dynamics: A Textile Framework For 3-graphs arXiv:2607.29233
Unverified 2026

Null-form quadratic wave layer

Replace an unconstrained quadratic interaction between channel derivatives with a learnable combination of Lorentzian and antisymmetric null forms. For wave-equation surrogates, this enforces exact cancellation when two interacting features have parallel null directions, suppressing resonant derivative products that otherwise cause unstable long-horizon rollouts.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Recovery of a Null Form in the Wave Equation from Scattering Data arXiv:2607.28917