Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

Unverified 2026

Leading-Term Strand Router

Build a sparse neural mixing layer from colored directed strands rather than a dense all-to-all matrix. Feature channels are assigned ordered colors, local trivalent junctions conserve every color, and an edge width is the weighted sum of the colors carried by that edge; a differentiable penalty favors monotone, crossing-free routings that define a canonical leading term. This creates a structured routing prior that can be compared directly against dense attention and unconstrained sparse…

Useful5/10
Difficulty6/10
Novelty7/10
Paper: Leading term strandings for webs arXiv:2608.27425
Unverified 2026

Boxicity-guided constraint attention

Replace an unconstrained pairwise attention score with an intersection of coordinate-wise threshold or interval compatibility heads. Each head is a supergraph that permits pairs satisfying one constraint, while the final attention edge exists only when every head permits the pair. This provides an interpretable inductive bias for multi-constraint relations and prevents the model from approximating a conjunction using a single unstable nonlinear score.

Useful5/10
Difficulty5/10
Novelty8/10
Paper: Boxicity and Threshold Dimension of Zero Divisor Graphs arXiv:2608.27381
Unverified 2026

Hyperbola-Tangent Quadratic Features

Add a bank of quadratic features encoding tangent contact with the reciprocal manifold x1 x2 = 1, rather than forcing a generic MLP to discover this interaction from arbitrary monomials. For positive bounded feature pairs, each feature is nonnegative and becomes exactly zero at a selected reciprocal operating point. The module can be used either as an input feature expansion or as a regularizer encouraging learned gates and scales to follow a reciprocal geometry.

Useful5/10
Difficulty3/10
Novelty7/10
Paper: Quadratic Convexification of a Square Truncated by a Hyperbola arXiv:2608.26639
Unverified 2026

Joint Bochner-Riesz Bilinear Graph Layer

Replace an unconstrained bilinear feature interaction with a joint spectral filter that only allows pairs of graph or spherical frequencies satisfying a soft radius constraint. The smooth factor attenuates interactions near and beyond the cutoff instead of making the hard low-pass decision used by ordinary spectral truncation, which should reduce high-frequency aliasing and unstable feature products.

Useful5/10
Difficulty6/10
Novelty6/10
Paper: Bilinear Bochner--Riesz Means on the Complex Sphere arXiv:2608.25884
Unverified 2026

Collision-free ternary attention

Represent a sparse higher-order attention head by a 3-uniform hypergraph whose hyperedge $(v,x,y)$ allows anchor token $v$ to aggregate a pairwise interaction between tokens $x$ and $y$. During mask construction, greedily reject edges that would create a $4$-cycle in the link graph $L(v)$, so the same pair of source tokens cannot reach an anchor through multiple redundant pairings. This produces a diversity-constrained sparse attention pattern with an explicit, measurable collision bound.

Useful5/10
Difficulty6/10
Novelty7/10
Paper: Regularity method for hypergraphs with $4$-cycle-free links arXiv:2608.24740
Unverified 2026

Möbius-compressed circular latent states

Represent a population of N circular latent states using a three-parameter Möbius transformation applied to fixed uniform reference phases, rather than learning N unrelated angles. The resulting states remain on the circle by construction and can model concentrated or nearly uniform phase populations through a single concentration parameter.

Useful5/10
Difficulty4/10
Novelty7/10
Paper: Unstable Manifolds for the Kuramoto Model: Convergence to the Ott-Antonsen Manifold arXiv:2608.24453
Unverified 2026

Polynomial Band-Pass Feature Mixer

Add a norm-controlled feature mixer that applies a polynomial spectral filter to the channel covariance of a transformer or MLP block. A quadratic filter centered at \(\rho\) suppresses covariance eigenmodes far from the target and preserves modes near it, providing a tunable alternative to purely variance-maximizing mixing or standard normalization.

Useful5/10
Difficulty5/10
Novelty6/10
Paper: Spectral Selection in Sphere-Constrained Flows Generated by Polynomials of the Dirichlet Laplacian arXiv:2608.24444
Unverified 2026

Branching-Pressure Router

Replace a generic MoE router entropy bonus with a branching-pressure objective that values routes according to both their stochastic entropy and their number of valid fine-grained continuations. The module can be implemented as a hierarchical router: a coarse state chooses a base transition, while a validity mask determines how many valid expert or latent branches lift that transition.

Useful5/10
Difficulty5/10
Novelty6/10
Paper: The entropy of Gromov-Thurston manifolds and branched coverings arXiv:2608.24220
Unverified 2026

Dimension-aware power-mean fusion

Use the paper's dimension-dependent exponent transformation to fuse nonnegative outputs from several branches. Instead of selecting an arbitrary generalized-mean exponent, choose the output exponent q=Q_d(p) induced by an input exponent p, making the fusion rule explicitly sensitive to the dimension of the barycentric variables.

Useful5/10
Difficulty3/10
Novelty5/10
Paper: Borell--Brascamp--Lieb inequality with finitely many output functions arXiv:2608.23963
Unverified 2026

Reliability-gated Laplacian positional encodings

Construct metric-graph Laplacian positional encodings only at frequencies whose empirical eigenvalues are statistically stable under the paper’s $(n v_\mu(h))^{-1/2}$ law. Use local ball-mass estimates and empirical eigengaps to gate or downweight unreliable eigenvectors, preventing small-sample spectral noise from entering a GNN or graph transformer.

Useful5/10
Difficulty4/10
Novelty5/10
Paper: Spectral stability of empirical metric-measure Laplacians arXiv:2608.23150
Unverified 2026

Fisher-Geometry Expert Menu

Construct a finite menu of experts or LoRA adapters by quantizing the oracle action manifold under a task-aware Fisher-information distortion rather than Euclidean parameter distance. The router can choose an installed expert after observing the input, but only k experts are stored or evaluated. The paper's frontier rate gives a falsifiable accuracy-versus-menu-size prediction.

Useful5/10
Difficulty6/10
Novelty4/10
Paper: Pre-Disclosure Experiment Menus: Oracle-Relative Risk and Joint Sample--Menu Asymptotics arXiv:2608.22905
Unverified 2026

Conservative amortized collision layer

Add a learned stochastic pair-interaction layer to a particle graph neural network, with a conditional normalizing flow generating the post-interaction relative state. Parameterize the update in center-of-mass and invariant relative coordinates so every sampled interaction preserves pair momentum and kinetic energy exactly. The flow learns the transition law directly from observed scattering or trajectory data, replacing repeated numerical collision solves or unconstrained message-passing…

Useful5/10
Difficulty6/10
Novelty6/10
Paper: A particle method for the Boltzmann equation via amortized sampling from Green's function of the lifted linear operator arXiv:2608.22880
Unverified 2026

S3-Holonomy Message Passing

Build a graph neural network on the dual graph of a triangulated surface whose messages are transported by \(\mathfrak{S}_3\) permutation matrices associated with adjacent-face color transports. This removes dependence on arbitrary local color-label choices and gives the network an explicit representation of noncontractible topology through holonomy around cycles.

Useful5/10
Difficulty5/10
Novelty6/10
Paper: Congruence classes of monodromies of even triangulations arXiv:2608.22814
Unverified 2026

Complex Phase-Corrected State Integrator

Use the complex-conjugate palindromic coefficient that cancels the leading temporal phase defect of oscillatory modes. Implement complex arithmetic directly or use an exactly equivalent doubled-real state, then project the final state to its real component for real-valued prediction tasks.

Useful5/10
Difficulty6/10
Novelty7/10
Paper: Sharp CFL stability and temporal-dispersion optimization of symmetric splitting schemes for time-domain Maxwell equations arXiv:2608.22315
Unverified 2026

Pole-Tuned Graph Residual Layer

Use the graph Laplacian spectrum to set the mixing and correction coefficients of a two-state graph-propagation block. Balancing the contraction of low-frequency consensus modes against high-frequency disagreement modes may reduce oversmoothing and make deep graph-neural networks less sensitive to manually selected residual coefficients.

Useful5/10
Difficulty6/10
Novelty5/10
Paper: Optimal Parameter Design for DIGing on Minimizing Unweighted Sum of Squares arXiv:2607.25463
Unverified 2026

Okada Non-Crossing Routing Attention

Replace dense token-to-token attention with a learned composition of adjacent routing operators whose connectivity is stored as a non-crossing labelled arc diagram. Canonicalize the composed routing program using the Okada relations, then execute only the surviving sparse token paths.

Useful4/10
Difficulty6/10
Novelty7/10
Paper: Diagrammatic Okada monoid and cellularity of the Okada algebra arXiv:2609.01440
Unverified 2026

Commutator-Polynomial Residual Adapter

Replace an unconstrained linear residual adapter by an operator \(T\) satisfying a polynomial relation in the commutator operator \(\Delta_A(X)=AX-XA\). Choose the polynomial roots in a stable half-plane so that repeated commutators become nilpotent, making repeated adapter application terminate algebraically and permitting a finite-polynomial inverse of \(I+T\).

Useful4/10
Difficulty6/10
Novelty9/10
Paper: Spectral Rigidity of Commutators: Dynamics, Resonance, and Nilpotency arXiv:2608.29574
Unverified 2026

Critical Interface State-Space Pooling

Add a fixed or weakly learned interface-localized branch to a sequence model. Set the critical mass term to zero and make the transport coefficient change sign across a learnable interface, producing a localized mode that pools information near a detected transition rather than averaging uniformly over the sequence.

Useful4/10
Difficulty5/10
Novelty8/10
Paper: Critical Topological Photonics in Synthetic Dimensions arXiv:2608.21791
Unverified 2026

m-Binomial Global Mixer

Insert a fixed or lightly gated lower-triangular binomial-transform layer into a sequence model to create global interactions across positions without forming attention logits. For a sequence of length N, mix each output position with all earlier positions using coefficients determined by an integer m; initialize the layer fixed and optionally learn a diagonal channel gate or a small mixture over m values.

Useful4/10
Difficulty5/10
Novelty8/10
Paper: $m$-Bell and $m$-Stirling numbers: Iterated binomial transforms, hyper-Bessel functions, and moments of the Conway--Maxwell--Poisson distribution arXiv:2608.12011
Unverified 2026

Multiplicity-free Schur channel tying

Tie neural parameters across feature channels according to the Schur multiplicity pattern of the fermionic coinvariant representation. In the two-fermion case, use one learned parameter block for each Schur degree instead of independently parameterizing every ordered pair of fermionic channels, eliminating redundant copies while preserving the relevant GL2 channel symmetry.

Useful4/10
Difficulty5/10
Novelty7/10
Paper: Type $B$ fermionic coinvariant rings arXiv:2608.02881
Unverified 2026

Rough Multiplicative Random Features

Replace explicitly stored independent random positional features with deterministic multiplicative phase features generated from a small table of random phases indexed by primes. Restrict positions to integers whose prime factors exceed a slowly growing threshold, because the paper's central-limit result says periodic weighted sums over these rough integers recover Gaussian behavior despite strong multiplicative dependence.

Useful4/10
Difficulty5/10
Novelty9/10
Paper: Random Multiplicative Functions with Periodic Weights arXiv:2608.00184
Unverified 2026

Coherent-Fluctuating Amplitude Units

Represent selected hidden features as z = sqrt(N) exp(i theta), with a persistent phase and an explicitly stochastic amplitude. Regularize the ratio between coherent power |E[z]|^2 and total power E[|z|^2] toward the condensate prediction pi/4, while optionally matching higher amplitude moments.

Useful4/10
Difficulty5/10
Novelty8/10
Paper: Coherent Bose-Einstein condensation with fluctuating density arXiv:2607.12926
Unverified 2026

Narayana-stable polynomial layer

Replace a monomial polynomial feature block by a fixed Narayana basis transformation. If the input polynomial has nonnegative coefficients and only real roots, the transformed polynomial is guaranteed to have only real roots as well, giving a certified stability-preserving coordinate change for polynomial neural networks.

Useful4/10
Difficulty5/10
Novelty9/10
Paper: The Narayana transformation arXiv:2607.01572
Unverified 2026

Reaction-Closed Sparse Routing

Represent the active experts or channels of a sparse layer by a presence set and impose a reaction-style dependency graph on possible support changes. During a growth phase, activate only the least support set closed under enabled dependencies; during later pruning, allow trajectory-dependent removals but never add structurally unreachable experts. This should reduce routing churn and dead experts while preserving adaptive sparsity.

Useful4/10
Difficulty5/10
Novelty7/10
Paper: A Structural Theory of Admissible Transitions in Biological Reaction Networks arXiv:2608.27201