Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

Unverified 2026

Delay-signature coincidence branches

Add a bank of temporal branches whose synapses apply heterogeneous delays before coincidence pooling. A branch selectively responds to an ordered pair or short sequence when the delay difference compensates the sequence interval, while low delay dispersion makes it primarily an order-invariant event detector. The branch threshold creates sparse, interpretable activations that can replace part of a recurrent or attention-based temporal module.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Axonal delay dispersion decides whether a neuron detects an event or a sequence, and predicts cortical column diameter arXiv:2609.04195
Unverified 2026

Resolution-Invariant Hilbert-Space CNF Prior

Replace a finite-dimensional latent prior for fields or trajectories with a continuous flow acting on function coefficients in a separable Hilbert space. A Gaussian reference with an explicit covariance spectrum is transformed by a learned vector field, so the model can represent smooth, non-isotropic function distributions while sharing parameters across discretization resolutions.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Learning Informative Prior with Infinite-Dimensional Continuous Normalizing Flow for Bayesian Inverse Problem arXiv:2609.03343
Unverified 2026

Uniform Species Architecture Sampler

Translate a recursive neural architecture grammar into a combinatorial specification and use its generating function to sample architectures uniformly conditional on size. This removes the severe shape and branching biases of naive random tree generation, producing a better-covered architecture-search population at the same evaluation budget.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Enumerating Chemical Structures, Stereoisomers, and Stereogenic Units with Analytic Combinatorics arXiv:2609.02932
Unverified 2026

Bicomplex Spatio-Temporal Residual Encoder

Replace independent per-sensor temporal encoders with a shared two-dimensional transform whose temporal and spatial phase channels remain algebraically distinguishable. Train a neural residual detector and localization head on the transformed coefficients, forcing perturbations with the same temporal signature but different positions to differ primarily in their spatial phase.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Multivariable Geometric Laplace Transform and Fault Detection in Distributed-Converter Lines arXiv:2609.02431
Unverified 2026

Signed de Bruijn parity memory

Augment an RNN or state-space model with a finite-state binary-context module whose transitions are those of a de Bruijn graph, while a signed transition channel records a quadratic parity function of the recent context. The exact finite-memory branch preserves cancellation-sensitive parity features that a continuous hidden state may forget, and a learned readout can combine it with the ordinary neural state.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: The Quadratic Easy Coefficients Conjecture via Finite-Type Shifts and Zeta Functions arXiv:2609.01399
Unverified 2026

Sparse Legendre Shape Hypernetwork

Use a sparse multivariate Legendre expansion as the geometry-to-network-weights map, rather than an unconstrained MLP that consumes all shape parameters. The hypernetwork predicts only coefficients for a selected set of polynomial multi-indices, allowing high-dimensional or countably parameterized shape uncertainty to be handled with a number of learned terms determined by coefficient decay rather than ambient dimension.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Shape Holomorphy and Sparse Approximation of the Maxwell Electric Field Integral Operator arXiv:2609.00466
Unverified 2026

Poisson Non-Backtracking Fingerprint

Add a fixed or lightly parameterized graph-level feature extractor based on short non-backtracking cycle counts, and concatenate its Poisson log-likelihood-ratio features with the output of a graph neural network. The feature scaling uses the paper's explicit means, so the network receives statistics that are approximately independent and correctly normalized rather than raw, highly correlated cycle counts.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Analysis of Polynomial Threshold Functions on Random Regular Graphs: Computational Complexity of Detecting Noisy Random Lift arXiv:2608.28539
Unverified 2026

Bernstein-Safe Policy Head

Replace a neural controller's pointwise action outputs over a finite horizon with Bernstein control points whose convex hull satisfies actuator and trajectory constraints. The network predicts the control points, while a robust margin accounts for bounded tracking or model-prediction error, making continuous-time actuator feasibility checkable from finitely many inequalities.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Actuator-Aware Spatiotemporal Tube Synthesis for Temporal Reach-Avoid-Stay Tasks arXiv:2607.23040
Unverified 2026

PCA-Discovered Implicit Latent Dynamics

Replace an explicit recurrent transition with a learned descriptor relation in latent space, allowing some latent coordinates to satisfy algebraic constraints rather than being numerically integrated. Fit the relation using total-least-squares or iterative PCA on the jointly observed trajectory, so noise in every channel is treated symmetrically and the model can discover whether the latent system is index-0 or index-1.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Subspace Based Identification of Errors-in-Variables Linear Descriptor Systems arXiv:2608.30259
Unverified 2026

Mass-Conserving Puncta Router

Insert a differentiable reaction-diffusion layer that converts dense token or pixel features into sparse, spatially coherent routing masks. Two competing orientations form complexes through conserved monomer reservoirs, so local assignments can cluster while opposite assignments mutually exclude one another instead of independently activating at the same location. The layer can be used as a soft-to-hard MoE router, attention-mask generator, or object-part grouping module.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Clustering versus sorting: a mass-conserving reaction-diffusion model of planar polarity puncta arXiv:2608.29679
Unverified 2026

Cumulative-Control Surrogate Encoder

Use cumulative control measures as the input to a neural surrogate instead of the full sequence of control values. For a quasi-static free-boundary system satisfying the paper's average-pressure path-independence assumption, two nonnegative control histories with identical integrals up to time t should produce the same state at t, allowing a smaller training input and fewer distinct control trajectories in the dataset.

Useful6/10
Difficulty4/10
Novelty8/10
Paper: Online Gate-Driven Flow Control in Resin Transfer Moulding Using a Neural-Network Surrogate arXiv:2608.29521
Unverified 2026

Stability-calibrated Sinkhorn attention

Replace independently normalized attention or routing weights with an entropic doubly stochastic transport plan, while choosing its regularization ε using the paper's explicit statistical-stability bound. Increase ε when residual inversion or minibatch fluctuations are amplified, and decrease it only when the estimated bound permits sharper assignments.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Uniform Statistical Convergence of Empirical Sinkhorn Potentials with Exponential and Polynomial Dependence on the Regularization Parameter arXiv:2608.29152
Unverified 2026

Conservative multifractal cascade prior

Use a conservative multiplicative cascade as a hierarchical latent prior or data-augmentation mechanism for models that generate intermittent, heavy-tailed, multiscale fields. The model receives a controllable cascade-width parameter, allowing systematic conditioning and evaluation across levels of non-Gaussianity instead of relying only on Gaussian latent noise.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Exact topology of conservative multiplicative cascades: An ultrametric transfer-operator genus arXiv:2608.24897
Unverified 2026

Leaf-Relocation Tree Router

Learn a branching hierarchy for tokens, examples, or experts by greedily relocating leaves to reduce T-Robinson violations. The resulting tree supplies hierarchical candidate sets for retrieval or MoE routing, allowing the model to search a small subtree instead of all items while adapting the hierarchy to learned representations.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: T-Robinson Spaces: Structure, Recognition, and Applications to Real Data arXiv:2608.21248
Unverified 2026

Max-times spectral erosion layer

Replace an unconstrained Fourier-domain linear mixer with a bank of positive spectral kernels and a max-times erosion aggregator. For a nonnegative Fourier magnitude f, each kernel produces a quotient response f/psi_k and the layer takes the pointwise supremum over kernels, giving exact positive homogeneity and monotonicity. This is most suitable as a drop-in spectral mixing block in a CNN, vision transformer, or state-space model.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Morphological Representation Theory in the Fourier Inf-Semilattice: Universal Decomposition of Frequency-Domain Deep Learning Operators arXiv:2608.20399
Unverified 2026

Sobolev-optimal frozen analytic features

Replace a trainable shallow hidden layer by a deterministic feature dictionary generated from Chebyshev-spaced scalar parameters and quasi-uniform ridge directions. Train only the output linear map, or use the frozen layer as the first stage of a larger network, thereby eliminating hidden-layer backpropagation while retaining a constructive smooth-function approximation guarantee.

Useful6/10
Difficulty4/10
Novelty6/10
Paper: Sharp Sobolev Approximation on General Domains by Linearized Shallow Networks with Analytic Activations arXiv:2608.18520
Unverified 2026

Parseval Scattering Stem with Certified Depth

Replace the first several convolutional blocks of a small image model with a finite-depth convolution-modulus scattering stem built from a Parseval filter bank. Enforce exact energy accounting and use the paper's polynomial residual law to choose the smallest depth that captures the desired fraction of input energy, avoiding unstable or redundant deep scattering paths.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Universal admissibility for scattering transforms arXiv:2608.18064
Unverified 2026

Dirac-Free Weak Mixed PINN

Build a neural PDE solver that predicts a regularized mixed flux rather than directly fitting a PDE residual containing a Dirac delta. Subtract the explicit radial field generated by the source and train the network with weak constitutive and conservation residuals, so the singularity is represented analytically instead of approximated by a narrow Gaussian.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Mixed Finite Element Methods for a Dirac Source: Divergence-Form Splitting and L^p Error Analysis arXiv:2608.17575
Unverified 2026

Dimension-calibrated contractive branch attractor

Construct a recurrent or generative network from finitely many contractive branches whose hidden-state attractor has a prescribed similarity dimension. The branch contraction ratios determine the target complexity through the equation sum_i r_i^s = 1, while a separation penalty approximates the open set condition and prevents branch collapse or excessive overlap.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: The Moran--Hutchinson formula in semimetric spaces arXiv:2608.16817
Unverified 2026

Negative Bethe-Hessian positional features

Augment node features with eigenvectors corresponding to negative eigenvalues of the Bethe-Hessian H(t,G), rather than using only Laplacian or adjacency positional encodings. The diagonal D-I correction is designed for sparse, locally tree-like graphs and should suppress degree-fluctuation artifacts near the connectivity threshold. Feed the resulting coordinates to a GNN through a learned gate so the model can ignore them when they are uninformative.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: The Bethe-Hessian down to the Percolation Threshold arXiv:2608.16672
Unverified 2026

Scalar Residual Thermodynamic Head

Attach a small temperature-pressure residual head to a pretrained structural encoder instead of relearning the full free-energy surface. Predict one scalar Gibbs free energy and obtain entropy, volume, and other thermodynamic responses by automatic differentiation, enforcing that all outputs derive from a common potential.

Useful6/10
Difficulty4/10
Novelty6/10
Paper: Universal Thermodynamic Interatomic Potentials for Crystalline Materials arXiv:2608.14502
Unverified 2026

Adaptive spectral-gap toroidal encoding

Replace ordinary absolute positional embeddings with coordinates on a learned flat torus and use dual-lattice Fourier characters as positional features. Control the covariance of the coordinate fundamental domain so that the paper's inequality guarantees a lower bound on the smallest nonzero positional frequency, preventing the learned periodic coordinate system from developing arbitrarily weak or nearly constant modes.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Spectral and Isoperimetric Bounds on Flat Tori arXiv:2608.13052
Unverified 2026

Resonance-Gated Triadic Fourier Layer

Replace unconstrained spectral mixing with a three-component triadic interaction whose strength is determined by the quadratic phase mismatch R(xi,xi_1). Near-resonant products receive high weight because their phases remain coherent, while strongly nonresonant products are attenuated. The resonance bandwidth can be fixed from the frequency grid or learned as a positive parameter.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: On solitary wave solutions with two-frequency parameters to the three-component system of quadratic nonlinear Schrödinger equations arXiv:2608.12983
Unverified 2026

Free-boundary neural halting

Replace a fixed confidence-threshold early-exit rule with a finite-horizon optimal-stopping policy over the model's evolving posterior confidence. The controller stops when the calibrated expected terminal error is no greater than the cost plus expected value of executing another neural block, permitting time-dependent and nonmonotone stopping regions.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: When should one stop the most exciting game? Sequential Inference for win-martingales arXiv:2608.12291