Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

Unverified 2026

Born-Structured Bilinear Neural Operator

Build each nonlinear correction in an inverse neural operator from explicit bilinear products of learned operator features, following the inverse Born expansion instead of using an unconstrained pointwise MLP. Use a square activation to implement multiplication exactly, and truncate the interaction order so the model has a controllable polynomial structure.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Inverse Born series based neural operators arXiv:2608.18262
Unverified 2026

Certified Component Projection for Learnable Graphs

Insert a projection step after a graph neural network proposes edge weights, replacing the proposed Laplacian by the closest valid Laplacian with a prescribed block-component structure. The projection removes cross-block interactions while minimally changing within-block weights, and a block spectral-gap constraint guarantees that each block is connected rather than accidentally splitting into smaller components.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Nearest Graph Laplacians with Prescribed Connected Components: A Convex Framework for Network Reconstruction arXiv:2608.18128
Unverified 2026

Parseval Scattering Stem with Certified Depth

Replace the first several convolutional blocks of a small image model with a finite-depth convolution-modulus scattering stem built from a Parseval filter bank. Enforce exact energy accounting and use the paper's polynomial residual law to choose the smallest depth that captures the desired fraction of input energy, avoiding unstable or redundant deep scattering paths.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Universal admissibility for scattering transforms arXiv:2608.18064
Unverified 2026

Probability-Preserving Zonotopic Neural Uncertainty

Attach a finite mixture of zonotopes to each uncertain neural input or hidden state, and propagate every mixture component through affine layers and conservative nonlinear relaxations. When the number of components grows, merge components only with an enclosing zonotope and sum their probability masses, preserving a formal lower bound on the probability that the true activation lies in the represented set.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: The Zonotopic Mixture Filter arXiv:2608.17897
Unverified 2026

Free-semigroup Toeplitz layer

Represent hierarchical or tree-structured hidden states on words over d symbols and replace a dense mixing layer by a noncommutative Toeplitz operator composed of shared word shifts. Coefficients are reused at every tree location, so the parameter count depends on maximum interaction depth rather than the number of nodes; an optional spectral penalty controls the amplification profile of finite-depth truncations.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Limiting eigenvalue distribution and entropy of multi-Toeplitz matrices arXiv:2608.17859
Unverified 2026

Hardy–Szegő Repulsive Token Router

Replace independent top-k token selection by a quality-weighted determinantal subset objective based on the Hardy–Szegő kernel. Tokens with high learned quality are preferred, but geometrically redundant tokens have a small determinant contribution, encouraging diverse sets of routed experts, retrieved items, or attended context tokens.

Useful6/10
Difficulty6/10
Novelty5/10
Paper: Hardy-Szegő Point Processes: Large Deviations and Strong Szegő Asymptotics arXiv:2608.17509
Unverified 2026

Euler-Hankel PSD Gram Activation

Replace an unconstrained entrywise nonlinearity on a positive Gram or covariance matrix by a learned scalar function satisfying the paper's finite-order positivity-preserver conditions. The transformed matrix remains PSD for matrices of the target width n, allowing nonlinear Gram propagation without eigenvalue clipping or projection.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: A finite-order characterization of entrywise positivity preservers arXiv:2608.15904
Unverified 2026

Leibenson Nonlinear Diffusion Layer

Replace a standard graph-convolution propagation step with a short time integration of the nonlinear graph flow \(\partial_t u=\Delta_p(u^q)\). The pointwise power \(q\) and gradient exponent \(p\) create state- and edge-gradient-dependent propagation: small signals can be suppressed or amplified by \(q\), while large graph discrepancies receive nonlinear diffusion controlled by \(p\). Use nonnegative feature states and conservative edge fluxes so the layer inherits positivity and total-mass…

Useful6/10
Difficulty4/10
Novelty6/10
Paper: Leibenson's equation on graphs arXiv:2608.15168
Unverified 2026

Quadratic-budget Toeplitz long-range layer

Replace a dense translation-invariant interaction matrix with a positive-definite Toeplitz kernel K_n(e^f) whose log-spectrum is parameterized by a small number of Fourier coefficients with 1/|k| decay. Use the paper's explicit quadratic term as a spectral-volume budget, allowing long-range structure while discouraging uncontrolled determinant growth and ill-conditioning. Subtracting this term from a log-determinant regularizer leaves a residual intended to capture higher-order deviations from…

Useful6/10
Difficulty6/10
Novelty7/10
Paper: On Toeplitz determinants with slow Fourier decay arXiv:2608.13182
Unverified 2026

Adaptive spectral-gap toroidal encoding

Replace ordinary absolute positional embeddings with coordinates on a learned flat torus and use dual-lattice Fourier characters as positional features. Control the covariance of the coordinate fundamental domain so that the paper's inequality guarantees a lower bound on the smallest nonzero positional frequency, preventing the learned periodic coordinate system from developing arbitrarily weak or nearly constant modes.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Spectral and Isoperimetric Bounds on Flat Tori arXiv:2608.13052
Unverified 2026

Cactus-Graph Phase Budgeting

Build neural computation graphs with explicitly phase-budgeted serial and parallel branches, treating serial compositions as SRG products and parallel residual branches as SRG sums. Allocate phase centers theta_i so that every loop or branch aggregate stays away from -1, enabling stability-aware architecture search and constructive control of branch gains.

Useful6/10
Difficulty7/10
Novelty8/10
Paper: The $θ$-Symmetric SRG with Applications to Stability of Cactus Dynamic Networks arXiv:2608.12591
Unverified 2026

Primitive-Fock Interaction Layer

Replace an unconstrained q-way polynomial or tensorized feature layer with separate decomposable and primitive interaction channels. The decomposable channel models interactions explainable as products of lower physical-weight feature blocks, while the primitive channel captures residual factors that cannot be represented by those products. This should reduce redundant high-order parameters and provide a controllable inductive bias for compositional or disentangled representations.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Weak Limits of Wiener Chaos: Primitive-Fock Classification and Hilbert-Stein Extraction arXiv:2608.12492
Unverified 2026

Ambiguity-Flat Weyl Feature Layer

Replace a random cyclic filter bank or patch projection with the Weyl–Heisenberg orbit of one normalized learnable prototype. Regularize the prototype so that all nonzero shift and modulation correlations have a large and nearly equal magnitude, maximizing the smallest eigenvalue of the induced feature Gram matrix and preventing poorly observed feature directions.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Uniformly Stable Minimal Weyl--Heisenberg Measurements Approaching the SIC Benchmark arXiv:2608.11850
Unverified 2026

Smith-Reduced Chain Encoder

Replace an ordinary hierarchical graph encoder with a finite chain-complex encoder whose learned boundary maps satisfy \(\partial_{k-1}\partial_k=0\). Compute Smith normal form on the integer incidence matrices and treat unit-labelled cell pairs as refinement overhead: cancel or gate those pairs before message passing, while preserving non-unit labels that encode genuinely nontrivial structure. The resulting representation should be insensitive to arbitrary cell subdivision while retaining…

Useful6/10
Difficulty6/10
Novelty6/10
Paper: A Chain- and Diagram-Level Semantics for Morphological Calculus Refinement, monodromy, and bivector orbit decompositions arXiv:2608.11325
Unverified 2026

Measure-Lifted Entropy Amplifier

Replace a deterministic latent state with a probability measure over latent states, represented by particles or weighted prototypes. Apply the learned latent transition to every particle, so one base trajectory map induces a dynamics on distributions; use an entropy-preservation or entropy-growth regularizer to prevent collapse of the ensemble. The mechanism predicts that any positive base-state trajectory entropy can generate unbounded distinguishability in the ideal measure space through…

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Entropies of compact subsets and supported measures arXiv:2608.09702
Unverified 2026

Scalene Nilpotent-Symmetry Network

Augment a sequence network with a learned staggered matrix-product-operator symmetry and penalize its commutator with the network map. Unlike ordinary equivariance, the auxiliary operator need not define a self-commuting transfer-matrix family: it can be discovered through cross-commutation with a second alternating operator, while nilpotency supplies a finite hierarchy of symmetry constraints. The model should preserve generalized symmetry sectors and exhibit lower commutator error on…

Useful6/10
Difficulty7/10
Novelty8/10
Paper: Scalene Yang--Baxter triples as a source of hidden symmetries beyond the ordinary Yang--Baxter equation arXiv:2608.09081
Unverified 2026

Single-Node Observable Leaky-RNN

Construct a sparse recurrent network with positive edge weights and Leaky-ReLU updates so that one selected hidden node, observed over a finite time window, contains enough information to reconstruct the full hidden state. Add an auxiliary decoder from the observed trajectory to the initial state or current state, and use graph rewiring or edge-growth until every hidden node has a directed path to the sensor within the observation horizon.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: On the Observability and Controllability of Leaky-ReLU Networks arXiv:2608.09059
Unverified 2026

Sobolev-Orthogonal MLP Features

Replace raw polynomial or Fourier-like features in a small MLP with basis functions orthonormal under a Sobolev inner product that jointly measures feature magnitude and input derivative magnitude. This explicitly controls feature smoothness while preserving decorrelation, potentially improving conditioning and reducing the need for large derivative-regularization coefficients.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: A Riemann-Hilbert representation for Sobolev orthogonal polynomials arXiv:2608.08397
Unverified 2026

Granularity-Aware Feasible Routing

Replace a continuous allocation or routing decision with a lattice-valued decision whose unit size is explicitly normalized by total capacity. Round allocations downward rather than to the nearest lattice point, preserving per-example capacity feasibility, and train or evaluate against the resulting granularity ratio rather than treating discretization as an implementation detail.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Bid Lattices and the Value of Flexibility:A Granularity Ratio for Capacity Markets arXiv:2608.08371
Unverified 2026

Maximal multiscale differential block

Replace a conventional feature-pyramid sum by a bounded multiscale differential transform. At each scale, subtract a blockwise conditional expectation from a local average, then combine these residuals with bounded coefficients. Add a penalty on the largest interval response so that contributions from adjacent scales cannot accumulate destructively or explosively.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Noncommutative maximal differential transforms associated to averaging operators arXiv:2608.07300
Unverified 2026

Monotone spectral activation

Replace an unconstrained matrix nonlinearity on small symmetric feature blocks with the isotropic spectral lift of a permutation-equivariant monotone map on eigenvalues. The layer remains orthogonally equivariant, while the paper's equivalence transfers a scalar inner-product monotonicity certificate from eigenvalue space to the full matrix space.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Monotonicity of isotropic tensor functions on the set of symmetric matrices: completing Rodney Hill's generalization of the Chandler Davis convexity theorem arXiv:2608.07087
Unverified 2026

KPZ Directed-Polymer Attention

Replace independent Gaussian attention noise or unconstrained token routing with a directed-polymer path distribution over positions and layers. The router aggregates exponentially many monotone paths through temporally correlated random edge scores, producing heavy-tailed but spatially coherent routing and preventing attention from collapsing onto a single token. The paper's t^{2/3} wandering and t^{1/3} free-energy fluctuations become measurable diagnostics and tunable targets rather than…

Useful6/10
Difficulty5/10
Novelty7/10
Paper: KPZ Superdiffusion of Local Correlators in Diffusive Random Quantum Circuits arXiv:2608.06459
Unverified 2026

LKJ Covariance for Variational Adapter Blocks

Use an LKJ correlation factor as the correlation component of a variational posterior over a compact adapter, LoRA factor, or Bayesian neural-network parameter block. The model learns marginal scales separately while the correlation matrix remains automatically positive semidefinite and unit-diagonal, avoiding unconstrained covariance matrices, invalid correlations, and fragile covariance decompositions.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Bartlett Couplings of the Onion and Vine LKJ Samplers arXiv:2608.06116
Unverified 2026

Braess-aware graph rewiring

Use Kemeny’s constant as a diffusion-quality gate when adding shortcut edges or cliques to a graph used by a GNN. Candidate augmentations are accepted only when they reduce estimated average hitting time, preventing rewiring operations that superficially shorten paths but make the random walk mix more slowly.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Kemeny's constant and Braess cliques in graphs arXiv:2608.04150