Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

Unverified 2026

Period-Resolvent Recurrent Layer

Construct a recurrent or graph-neural layer on a finite state space with a known bijection T, such as a modular cat map, and use the diagonal resolvent gain (1 − α^kx)^−1 as a state-dependent self-return or memory coefficient. States on short periodic orbits receive larger amplification, while long-period states receive weaker amplification, producing deterministic localization without learned disorder.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Arithmetic Landscape Functions of a Discrete Cat Map arXiv:2607.24857
Unverified 2026

Hypertree-Sparse Tensor Initialization

Replace a dense order-p tensorized neural layer with a sparse random hypergraph tensor whose support probability is alpha = c/N^{p-1}. Assign independent zero-mean marks to active hyperedges, so every tensor moment scales as N^{-(p-1)} while the total variance entering each output remains controlled. The resulting locally tree-like computation graph should reduce parameters and multiply-adds without immediately destroying signal propagation.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Spectral aspects of random heavy-tailed tensors arXiv:2607.24695
Unverified 2026

Connectivity-aware ATC gradient tracking

Use the paper's mode decomposition to choose between ordinary DIGing and augmented ATC-DIGing/AugDGM according to the communication spectrum and curvature. The augmented scheme changes the disagreement feedback gain mode by mode, which can reduce the dominant pole radius on well-connected graphs without changing the neural-network architecture.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Exact Worst-case Convergence Rates of Distributed Gradient Tracking Methods arXiv:2607.23601
Unverified 2026

Imaginary-Axis Gramian Compression for Neural SSMs

Replace a large stable linear state-space or recurrent layer by a lower-order balanced realization computed from frequency-targeted controllability and observability Gramians. Use generalized low-rank ADI with imaginary-axis shifts concentrated at frequencies that dominate the training data, then retain states associated with the largest approximate Hankel singular values. This should reduce recurrent inference cost while preserving the layer's input-output response in the selected frequency…

Useful6/10
Difficulty6/10
Novelty7/10
Paper: A New Low-Rank Cholesky-Factor ADI Algorithm Allowing Shifts Anywhere in the Complex Plane with Applications to Data-Driven Model Reduction arXiv:2607.21969
Unverified 2026

Entropy-Calibrated Non-Backtracking Message Passing

Replace ordinary graph propagation, which repeatedly revisits the edge it just traversed, with a directed-edge non-backtracking operator. Normalize its learned gain using an estimate of the Hashimoto spectral radius so that feature magnitudes neither explode on high-growth graphs nor vanish on sparse graphs.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Critical-exponent spectra and rank two inverse realization on biregular trees arXiv:2607.21294
Unverified 2026

Degree-Capped Simplicial Residual Step

Set the residual propagation coefficient of a simplicial neural layer from a cheap upper bound on the operator spectrum instead of tuning it blindly. The degree-majorization theorem supplies a bound on the largest eigenvalue, while the Brouwer-type inequality supplies a topology-count-based bound on sums of the top eigenvalues.

Useful6/10
Difficulty3/10
Novelty6/10
Paper: Degree Majorization and Laplacian Eigenvalue Sums for Simplicial Complexes arXiv:2607.20910
Unverified 2026

Slepian-Concentration Feature Initialization

Initialize a coordinate-network feature bank with the leading eigenfunctions of a bandlimited concentration operator instead of random Fourier features. For a desired spatial region E, these features maximize the fraction of their L2 energy inside E among all functions with frequency support in Omega, giving a principled basis for localized signals.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Optimal concentration in the Paley-Wiener space arXiv:2607.19192
Unverified 2026

Volume-Weighted Hodge Convolution

Replace the ordinary combinatorial Hodge propagation in a simplicial neural network with a geometry-induced weighted Hodge Laplacian built from Euclidean simplex volumes. The operator preserves the harmonic/topological subspace while changing the positive spectrum according to the shape and scale of the simplices, allowing message passing to distinguish geometrically meaningful cells that have identical incidence patterns.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Geometry-Induced Hodge Stars on Rips and Dowker--Rips Complexes arXiv:2607.18692
Unverified 2026

Ramanujan Signed Ring Mixer

Replace an unsigned two-hop cyclic mixer by the paper's alternating signed circulant. The sign pattern preserves one-step and two-step interactions while reducing the exact spectral radius from 4 to 2√2, allowing a larger raw mixing coefficient under the same operator-norm stability constraint.

Useful6/10
Difficulty3/10
Novelty6/10
Paper: Signed circulants at the Ramanujan bound arXiv:2607.18334
Unverified 2026

KS-Balanced Spectral Residual Block

Replace or augment a residual neural layer with a Fourier-domain scale-selective flow containing a learned second-order term and a fourth-order stabilizer. The block permits controlled low-frequency amplification, as required by the KS infrared mechanism, while damping high-frequency feature noise and preventing unbounded spectral growth.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Large scale behavior in the Kuramoto-Sivashinsky equation: The Schwinger-Dyson route arXiv:2607.17915
Unverified 2026

Anisotropic Anharmonic Diffusion Layer

Insert a learnable semigroup layer that evolves features according to a positive operator combining frequency damping and spatially varying confinement. Unlike isotropic Gaussian smoothing, the layer can damp selected frequencies differently along different axes and can suppress activations in learned spatial regions, while the positive-semigroup construction prevents amplification.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Sharp Time-Decay Estimates for Fractional Heat Semigroups Associated with Polynomial Anharmonic Oscillators arXiv:2607.17580
Unverified 2026

Parity-Constrained Signed Propagation

Replace the unsigned adjacency used by a deep message-passing network with a signing selected from an affine family that makes designated short even cycles unbalanced. Search this family for a small even-power trace, which acts as a proxy for a smaller spectral radius and suppresses explosive long-range propagation. The signing can be fixed before training, so the method adds no per-example inference cost.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Parity families and a kernel-averaged L-function for near-Ramanujan signings arXiv:2607.17343
Unverified 2026

Dyson Diagonal Scaling for Directed Message Passing

Replace ordinary row-degree or symmetric normalization in a directed graph neural network with a nonlinear Dyson scaling. For a nonnegative directed adjacency matrix A, solve a positive vector equation and propagate with B = D A D, where D is the diagonal matrix of the solution. The resulting operator has row sums strictly below one, giving an explicit bound against exploding directed message propagation while retaining asymmetric edge information.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Non-symmetric vector dyson equations arXiv:2607.16333
Unverified 2026

Line-Graph Spectral Edge Parameterization

Replace one independently learned vector per graph edge with a truncated spectral expansion on the line graph. The model learns coefficients for low-frequency edge modes and reconstructs edge features before message passing, reducing parameters while imposing an inductive bias that incident edges should have correlated behavior.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Lossy compression of weighted graph adjacency matrices by transform coding arXiv:2607.14834
Unverified 2026

Spectral latent geometry for sparse attention

Build a sparse graph by thresholding normalized token or item inner products, then use the leading eigenvectors of its centered adjacency matrix as geometric features or a low-rank attention-logit bias. The graph avoids storing all pairwise similarities, while the paper's spectral bound supplies a concrete signal-to-noise test for deciding whether the resulting embedding is trustworthy.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Spectral Concentration and Recovery in Sparse High-Dimensional Random Geometric Graphs arXiv:2607.14304
Unverified 2026

Hermite-Schatten spectral layer

Replace a dense learned linear operator on continuous or image features by a truncated Hermite projection expansion whose coefficients are directly regularized in a Schatten-p norm. The layer becomes a structured low-rank operator, while the radial Hermite-Laguerre correspondence provides an analytically tractable parameterization and an exact spectral penalty.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Quantitative Fourier Restriction Estimates for Weyl Operators: Fourier-Support Dependence and Lower Bounds arXiv:2607.13697
Unverified 2026

Analytic KL spatial adapter

Replace a dense spatial parameter field in a neural field or convolutional adapter by a truncated squared-exponential KL expansion with analytic Gaussian-Hermite modes. The amplitude and correlation length remain trainable, but changing them only rescales coefficients and basis parameters instead of triggering a numerical eigensolve.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Hierarchical Bayesian inversion using the Karhunen-Loève expansion with analytical eigenpairs of the squared exponential kernel arXiv:2607.12387
Unverified 2026

Residual-Update Halting

Replace activation-magnitude-based adaptive computation halting with a criterion based on the actual recurrent update and a local stability margin. The loop halts when the state change is small relative to state scale for several consecutive steps, avoiding pathological decisions when LayerNorm-driven dynamics cause the activation norm to collapse.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: LayerNorm as Implicit Gain Control in Looped Transformers arXiv:2607.10681
Unverified 2026

Block-Probed Rational Spectral Layer

Replace a polynomial graph filter or repeated matrix multiplications in a graph neural network with a small rational filter evaluated at several shifts. Treat the incoming feature matrix as a block of probes rather than processing scalar probe vectors independently, allowing one set of shifted solves to expose multiple spectral directions simultaneously. The expected gain is higher approximation quality at the same number of operator applications, especially when the target filter has sharp or…

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Convergence analysis of a nonlinear eigensolver based on rational approximation of the resolvent arXiv:2607.10377
✗ Mechanism failed 2026

Resolution-adaptive spectral front end

Replace a fixed Fourier or spectral resolution in a neural operator or sequence model with a data-adaptive spectral cutoff. Keep only modes whose estimated signal energy exceeds the noise-amplification and discretization floor implied by the available number of trajectories and samples per trajectory. This should reduce overfitting to high-frequency sensor noise and preserve accuracy when the same model is deployed at different sampling resolutions.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: The Cost of Discretization in Functional Linear Regression: Minimax Rates and Adaptation arXiv:2607.09350
△ Mechanism confirmed, baseline not beaten 2026

Spectral Hamiltonian Neuron

Replace a scalar neuron activation with a matrix function of a learned Hamiltonian. Fixed Hermitian interaction operators are combined as a trainable linear Hamiltonian, the activation is applied to its eigenvalues, and the resulting observable is measured on an input quantum state. Noncommuting interaction terms provide a controlled source of expressivity beyond an ordinary scalar neuron.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Canonical quantization of neurons arXiv:2607.05000
✓✓ Beats tuned baseline 2026

Observability-Gated Spectral Phase Initialization

Add a preprocessing or differentiable synchronization layer that estimates one unit-modulus complex phase per graph node or data view from noisy pairwise relative-phase observations. Initialize the phases with a leading-eigenvector method, fix the global phase gauge, and allow nonlinear refinement only when the estimated perturbation is small relative to the observable Jacobian margin. This replaces random initialization for rotation-alignment modules and should reduce bad local minima caused…

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Spectral Initialization and Certification for Power System Angle Estimation arXiv:2607.06762
✗ Mechanism failed 2026

Periodic CMV Unitary Recurrent Layer

Replace a dense recurrent transition matrix with a periodic CMV-style product of alternating local 2x2 unitary cores. The transition is exactly norm-preserving, has O(n) trainable parameters under periodic tying, and can be applied through local factor operations rather than stored as an n-by-n matrix. Use turnover refactorization when changing the ordering or boundary connection of cores, enabling a compact cyclic unitary state-space layer.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Fast computation of eigenvalues of periodic CMV matrices arXiv:2607.06400
✓✓ Beats tuned baseline 2026

Critical-Tail Multiscale Mixer

Add a fixed or weakly parameterized residual mixer whose interaction between sequence positions at distance \(r\) is proportional to \(1/(r\log^2 r)\). Instead of truncating the kernel at a short radius, represent its heavy tail with dyadic distance bands and compute each band using prefix sums or block pooling, giving every token access to arbitrarily distant context at roughly \(O(L\log L)\) cost.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Long-range interactions and Anderson localisation for one-dimensional high-contrast resonator chain arXiv:2607.04971