Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

Unverified 2026

Bicomplex Spatio-Temporal Residual Encoder

Replace independent per-sensor temporal encoders with a shared two-dimensional transform whose temporal and spatial phase channels remain algebraically distinguishable. Train a neural residual detector and localization head on the transformed coefficients, forcing perturbations with the same temporal signature but different positions to differ primarily in their spatial phase.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Multivariable Geometric Laplace Transform and Fault Detection in Distributed-Converter Lines arXiv:2609.02431
Unverified 2026

Signed de Bruijn parity memory

Augment an RNN or state-space model with a finite-state binary-context module whose transitions are those of a de Bruijn graph, while a signed transition channel records a quadratic parity function of the recent context. The exact finite-memory branch preserves cancellation-sensitive parity features that a continuous hidden state may forget, and a learned readout can combine it with the ordinary neural state.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: The Quadratic Easy Coefficients Conjecture via Finite-Type Shifts and Zeta Functions arXiv:2609.01399
Unverified 2026

Marginal Fractional Coupling Layer

Replace a local smoothness penalty or local state transition along a sequence or depth coordinate by a marginal fractional quadratic energy with Fourier multiplier |k|. The sigma=1 kernel is nonlocal and scale-free, so it can preserve long-range correlations while suppressing high-frequency instability more selectively than an ordinary Laplacian penalty.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: BKT-like Correlation Scaling and Twist Responses in a One-Dimensional Fractional $U(1)$ Ginzburg--Landau Model arXiv:2609.00721
Unverified 2026

Critical Heavy-Tail Propagation Layer

Insert a sparse heavy-tailed nonlocal mixing operator into a residual sequence, graph, or spatial network so that information can traverse distant positions without stacking many local layers. Use the critical tail exponent s = 1/2, whose truncated first moment grows logarithmically and predicts an effective propagation distance proportional to depth times log depth rather than merely depth.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Propagation rates in integro-differential equations of ignition type arXiv:2608.28450
Unverified 2026

Transverse Quenched Feature Flow

Add a fixed, spatially correlated perturbation field to every layer of a CNN or 2D state-space model, with the perturbation decomposed into transverse and longitudinal Fourier components. Unlike ordinary injected noise, the same field is reused for all training examples and all forward passes, allowing it to act as a structured architectural flow that can promote global feature alignment. Sweep the transverse fraction at fixed total perturbation variance and test for the predicted ordering…

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Correlated disorder versus correlated noise: Ordering in active systems arXiv:2608.28012
Unverified 2026

Trainable two-sided activation gate

Replace the dense output of selected linear projections with a two-sided magnitude threshold that emits zero for small values but preserves signed large values. Learn one positive threshold per projection, or optionally one threshold per output channel, so the network discovers where sparse events can be removed while retaining outlier information.

Useful6/10
Difficulty4/10
Novelty5/10
Paper: Event-Driven Language Models with Sparse Neural Activity for Neuromorphic Hardware arXiv:2608.30439
Unverified 2026

PCA-Discovered Implicit Latent Dynamics

Replace an explicit recurrent transition with a learned descriptor relation in latent space, allowing some latent coordinates to satisfy algebraic constraints rather than being numerically integrated. Fit the relation using total-least-squares or iterative PCA on the jointly observed trajectory, so noise in every channel is treated symmetrically and the model can discover whether the latent system is index-0 or index-1.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Subspace Based Identification of Errors-in-Variables Linear Descriptor Systems arXiv:2608.30259
Unverified 2026

Composite-cycle interface search

Use exhaustive finite-state analysis as an architecture-search objective for coupled recurrent motifs. Instead of independently tuning a normalization gate and a WTA gate, enumerate their possible interfaces and select couplings that create group action only on a small joint image set, yielding controlled reversible subdynamics embedded in an otherwise dissipative system.

Useful6/10
Difficulty6/10
Novelty9/10
Paper: "More Is Different'' in Neural Circuits: Algebraic Emergence of Effective Theories in Canonical Recurrent Motifs of Biological Neuronal Networks arXiv:2608.30231
Unverified 2026

Relative-Degree-Gated Passive Neural State Space

Construct a neural state-space model with an explicit first-order input-to-output path instead of forcing every output to depend only on deeply propagated hidden states. Penalize or reject learned linearizations whose transfer matrix has relative degree greater than one, then train a storage-function certificate for the remaining passive dynamics. This preserves the paper's relative-degree compatibility condition while allowing high-order internal memory.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Relative-Degree Wall Restricts Passivity-Based Stability Analysis in Inverter-Dominant Grids arXiv:2608.29474
Unverified 2026

Laguerre-Optimal Positive Delay Filter

Replace an Erlang delay or exponential smoothing cascade in a recurrent or state-space layer by a positive rational kernel of the form \(\kappa(u)=C e^{-a u}p(u)^2\). Choose the degree-\(m\) polynomial by deleting the adjacent pair of Laguerre zeros with smallest relative gap from \(L_{m+2}\), then rescale the resulting density to unit mean. This preserves a nonnegative impulse response while reducing temporal jitter relative to Erlang filters.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Least Variability in a Polynomial-Square Class of Rational Kernels arXiv:2608.29143
Unverified 2026

Sparse Levy Skip Network

Construct a residual neural network or graph message-passing layer whose skip edges are sampled with probability proportional to their distance as $|i-j|^{-(1+\sigma)}$, while retaining a small local backbone. The paper's mechanism predicts that coarse-grained propagation is governed by the long-range kinetic operator and is therefore asymptotically insensitive to the particular Bernoulli graph realization, yielding controllable superdiffusive information transport without dense all-to-all…

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Universality of superdiffusion in simple random graphs arXiv:2608.23207
Unverified 2026

Max-times spectral erosion layer

Replace an unconstrained Fourier-domain linear mixer with a bank of positive spectral kernels and a max-times erosion aggregator. For a nonnegative Fourier magnitude f, each kernel produces a quotient response f/psi_k and the layer takes the pointwise supremum over kernels, giving exact positive homogeneity and monotonicity. This is most suitable as a drop-in spectral mixing block in a CNN, vision transformer, or state-space model.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Morphological Representation Theory in the Fourier Inf-Semilattice: Universal Decomposition of Frequency-Domain Deep Learning Operators arXiv:2608.20399
Unverified 2026

Bures-Shaped Latent State Space

Add a stable linear latent state-space block whose controllability Gramian is trained toward a chosen positive-definite target using squared Bures–Wasserstein distance. Direction-specific semidefinite constraints can suppress disturbance amplification in nuisance coordinates while preserving controllability in coordinates needed for prediction.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: A Controllability Gramain Shaping with LMI Constraints under Bures--Wasserstein Distance arXiv:2608.19754
Unverified 2026

Criticality-aware fractional drift block

Replace an unconstrained multiscale residual block by the sum of a fractional diffusion branch and a drift or transport branch whose strength follows the PDE scaling law. At finer spatial scales, the drift coefficient is multiplied by R^{2s-1}; this suppresses unstable transport when s>1/2 while preserving equal-strength diffusion and drift at the critical value s=1/2.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Gradient regularity and potential estimates for fractional drift--diffusion equations in the critical and subcritical ranges arXiv:2608.19571
Unverified 2026

Normal Spectral Linear Layer

Replace an unconstrained dense transition or recurrent matrix with a normal matrix $A=U\operatorname{diag}(\lambda)U^*$, where $U$ is unitary and $\lambda$ contains learnable eigenvalues. The layer can be initialized by fitting a normal matrix to input-output pairs through the paper's objective, then trained with Riemannian updates that keep $U$ unitary and preserve the normal-operator structure.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: The Normal Procrustes Problem: A Riemannian Optimization Approach arXiv:2608.19513
Unverified 2026

Integrable Elliptic Memory Cell

Replace an unconstrained recurrent transition with block-diagonal planar rotations whose angles are learned or conditioned on a slowly varying context variable. The resulting hidden-state norm and each two-dimensional block energy are exactly invariant in the ideal recurrence, preventing exploding or vanishing recurrent dynamics while retaining phase information over long horizons.

Useful6/10
Difficulty4/10
Novelty4/10
Paper: Outer Contact Billiards arXiv:2608.19393
Unverified 2026

BT-Critical Long-Memory Initialization

Construct a recurrent or state-space layer whose equilibrium Jacobian is placed near a nondegenerate Bogdanov–Takens point, then use a small unfolding parameter to move between damped, oscillatory, and slowly relaxing regimes. Unlike eigenvalue-only initialization near one, this controls both the double-zero center structure and the quadratic nonlinear coefficients that determine the local phase portrait.

Useful6/10
Difficulty6/10
Novelty8/10
Paper: The Bogdanov--Takens normal-form coefficients in $\mathbb{R}^n$ as directional derivatives of the characteristic invariants arXiv:2608.19018
Unverified 2026

Superperiodic Frieze Sequence Mixer

Replace dense token-to-token attention in a controlled ablation with a cyclic order-a recurrence whose coefficients are periodic across positions. The resulting mixer has linear cost in sequence length for fixed recurrence order and can enforce a fixed signed periodic boundary condition, providing a compact structured alternative to local attention or a lightweight state-space model.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Combinatorics of the Fourier transform: Stokes data, Gale duality and frieze patterns arXiv:2608.17992
Unverified 2026

Probability-Preserving Zonotopic Neural Uncertainty

Attach a finite mixture of zonotopes to each uncertain neural input or hidden state, and propagate every mixture component through affine layers and conservative nonlinear relaxations. When the number of components grows, merge components only with an enclosing zonotope and sum their probability masses, preserving a formal lower bound on the probability that the true activation lies in the represented set.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: The Zonotopic Mixture Filter arXiv:2608.17897
Unverified 2026

Data-Identified Neural Dependency Graph

Partition a neural state or feature vector into blocks and identify directed dependencies between blocks from one-step transition data. Use the inferred design structure matrix as a hard mask or soft gate on recurrent, state-space, graph, or mixture-of-experts couplings, replacing a dense unconstrained interaction matrix with a data-supported sparse graph.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Novel methodology for obtaining design structure matrices using network identification arXiv:2608.16759
Unverified 2026

Fractional-to-Local RG Residual Block

Replace a single local message-passing or convolution operator by a spectrally controlled mixture of fractional and ordinary diffusion. The exponent σ is learned or scheduled, while a crossover gate forces the model to change parameterization near the renormalization-group threshold σ*=2, allowing long-range propagation when useful without retaining an unnecessarily nonlocal operator at short scales.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: The $6-ε$ Expansion for Long-Range Lee--Yang and Percolation Criticality arXiv:2608.15120
Unverified 2026

Laguerre Memory Convolution

Replace the length-L learned convolution kernel in a causal sequence layer with K Laguerre basis functions, where K is much smaller than L and the basis parameter controls the decay time scale. The layer retains a long receptive field but learns only K coefficients, while FFT or a fixed state-space realization evaluates the resulting convolution efficiently. This is especially appropriate for audio, sensor streams, and long-context regression where the desired impulse response is smooth or…

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Impulse Response Estimation via Laguerre-Fourier Expansion arXiv:2608.14769
Unverified 2026

Transform-domain real packing

Pack two real tensors into the real and imaginary components of one complex tensor, perform one complex transform, and recover both real spectra through conjugate symmetry. Combine this with implicit padding so separate padded real arrays and transform workspaces are not allocated.

Useful6/10
Difficulty6/10
Novelty5/10
Paper: Hybrid Dealiasing and Implicit Packing for Real Convolutions arXiv:2608.14497
Unverified 2026

Quadratic-budget Toeplitz long-range layer

Replace a dense translation-invariant interaction matrix with a positive-definite Toeplitz kernel K_n(e^f) whose log-spectrum is parameterized by a small number of Fourier coefficients with 1/|k| decay. Use the paper's explicit quadratic term as a spectral-volume budget, allowing long-range structure while discouraging uncontrolled determinant growth and ill-conditioning. Subtracting this term from a log-determinant regularizer leaves a residual intended to capture higher-order deviations from…

Useful6/10
Difficulty6/10
Novelty7/10
Paper: On Toeplitz determinants with slow Fourier decay arXiv:2608.13182