Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

Unverified 2026

Schatten-Stable Noncommutative Matrix Layer

Replace an ordinary elementwise interaction between two feature matrices by a noncommutative functional-calculus layer \(\varphi(A,B)\), where \(A\) and \(B\) are Hermitian channel operators that need not commute. Add a soft penalty on \([A,B]=AB-BA\), and use a Besov-smooth parameterization of \(\varphi\) so that perturbations are controlled in Schatten \(p\)-norm for \(p\leq2\). This creates a principled matrix interaction module that can remain stable when feature operators or graph…

Useful5/10
Difficulty6/10
Novelty8/10
Paper: Commutator estimates for functions of noncommuting self-adjoint operators arXiv:2608.16731
Unverified 2026

Matroid-Capacity Router

Replace independent top-k expert decisions by a global fractional routing problem that enforces token-side and expert-side capacities together with an additional diversity constraint represented by a partition or laminar matroid. Use the resulting Hall-type deficiency certificate to identify overloaded token subsets and penalize the actual structural cause of routing failure rather than relying only on an aggregate load-balancing loss.

Useful5/10
Difficulty6/10
Novelty4/10
Paper: Measurable Matroids: Foundations and Min--Max Theorems arXiv:2608.16464
Unverified 2026

Hermitian ETF classifier head

Construct a classifier whose normalized class vectors form an explicit 2d-line equiangular tight frame instead of using independently initialized weights. The ETF gives every class the same norm, equal pairwise coherence, and an isotropic frame operator, which should make final-layer gradients better conditioned and reduce accidental class crowding. The classifier can be fixed, or restricted to a learned unitary rotation of the ETF so that its geometry is preserved during training.

Useful5/10
Difficulty4/10
Novelty4/10
Paper: New constructions of optimal arrangements of $2d$ lines in $\mathbb{C}^d$ arXiv:2608.16116
Unverified 2026

PSD-Safe Learnable Similarity Kernel

Use the finite-order characterization to learn a nonlinear similarity function for token, patch, or graph-node Gram matrices while preserving PSD by construction or by a differentiable certificate loss. This creates a kernelized attention or graph-readout mechanism in which nonlinear affinity transformations cannot introduce indefinite similarity geometry.

Useful5/10
Difficulty6/10
Novelty6/10
Paper: A finite-order characterization of entrywise positivity preservers arXiv:2608.15904
Unverified 2026

Singularity-isolated cell interaction layer

Replace pointwise pair interactions between mesh cells by quadrature of the interaction kernel over the full Cartesian product of the two cells. Decompose each cell pair into convex-hull pieces and apply a Duffy-like radial transformation so the coincidence singularity is confined to one quadrature coordinate, allowing fixed Gauss-Jacobi or adaptive quadrature to produce smooth, low-variance interaction features.

Useful5/10
Difficulty7/10
Novelty7/10
Paper: Space-Time Galerkin Boundary Element Method for the Wave Equation arXiv:2608.15292
Unverified 2026

Fingerprint-Aware Neural Graph Certificates

Represent a neural computation or verification pipeline as a directed acyclic graph whose nodes carry cached certificates for tensor shapes, numerical ranges, Lipschitz estimates, quantization error, or equivalence to a reference module. After locally replacing or optimizing one node, compare its old and new interface fingerprints and revoke certificates only along the dependency cone when the interface changed. This enables safe incremental verification during architecture search, compiler…

Useful5/10
Difficulty4/10
Novelty8/10
Paper: Andy: A Mathematical Agent for Rigorous Proof and Autonomous Research arXiv:2608.15052
Unverified 2026

Correlated-Gaussian Orbit Fingerprint

Replace a polynomial layer's single-replica output statistics with a finite fingerprint computed from several correlated Gaussian replicas. Train the fingerprint to be invariant under orthogonal reparameterizations while remaining discriminative between genuinely different polynomial maps, preventing models from collapsing distinct tensor functions that have identical marginal output laws. This is a practical symmetry-aware regularizer or auxiliary embedding for tensorized MLPs and polynomial…

Useful5/10
Difficulty5/10
Novelty8/10
Paper: Finite Gaussian Reconstruction of Polynomial Orbits: From Correlated Moments to Oscillatory Periods arXiv:2608.14475
Unverified 2026

Jordan spectral feature scaling

Group neural features into small Hermitian matrix elements and scale each group with the paper's tracial spectral Lp norm rather than independently normalizing scalar channels. This introduces a coupled spectral geometry while remaining implementable with ordinary eigendecompositions in the associative Hermitian-matrix special case.

Useful5/10
Difficulty5/10
Novelty8/10
Paper: Spectral nonassociative $\mathrm{L}^p$-spaces for $\mathrm{JBW}^*$-algebras arXiv:2608.14231
Unverified 2026

Velocity-Scaled Symbolic Flow Model

Represent a continuous-time neural dynamical system as a symbolic Markov chain over regions together with a positive learned roof function giving the time spent in each region. Weight local reconstruction and prediction errors by the predicted vector-field speed, following the paper's scaled Hölder coding relation, so that the model does not over-penalize arbitrarily small coordinate errors near equilibria. This produces a hybrid latent model with discrete long-range structure and continuous…

Useful5/10
Difficulty6/10
Novelty7/10
Paper: Symbolic dynamics for non-uniformly hyperbolic flows arXiv:2608.14095
Unverified 2026

Hardy-Basis Equivariant Mixer

Insert a fixed or partially learnable equivariant change-of-basis module into a spherical or SO(3)-equivariant network. At each angular frequency \(\ell\), the module maps the line selected by the line-bundle quantization to the line selected by the Grauert-tube quantization, allowing the network to represent both holomorphic/base-local and geodesic-flow-adapted features without breaking rotation equivariance.

Useful5/10
Difficulty5/10
Novelty8/10
Paper: Intertwining the line bundle and Grauert-tube Hardy quantizations of the round 2-sphere arXiv:2608.13965
Unverified 2026

Discrepancy-bounded signed fast memory

Replace an unrestricted additive recurrent or fast-weight memory with a sign-selectable update: for each incoming update vector, choose between adding and subtracting it so that a smooth compact potential of the memory state is minimized. This is appropriate when the memory representation has sign symmetry, such as signed random features or a learned linear sketch; it is not a drop-in replacement for ordinary gradient updates where the sign carries semantic information.

Useful5/10
Difficulty5/10
Novelty9/10
Paper: Online balancing of vectors with small coordinates arXiv:2608.12490
Unverified 2026

Mutation-invariant interaction mixer

Represent a directed interaction graph by a Laurent-polynomial Euler-like matrix and use its evaluation as a signed message-passing or attention-mixing operator. During dynamic rewiring, require the new graph representation to preserve the associated bilinear form up to the congruence transformation induced by the change of basis, so equivalent routings produce equivalent hidden states.

Useful5/10
Difficulty6/10
Novelty8/10
Paper: Monodromy of plane curve singularities and quiver mutation arXiv:2608.12484
Unverified 2026

Pisot-Separated Multiscale Codes

Replace an unconstrained geometric multiscale codebook by features generated from a finite digit set and a Pisot scale factor. The contracting algebraic-conjugate directions should suppress near-collisions between representations at different scales, producing a discretely separated hierarchy that can be used for embeddings, recurrent memory, or quantized transformer states.

Useful5/10
Difficulty6/10
Novelty9/10
Paper: Self-similar Delone sets and Pisot numbers arXiv:2608.11867
Unverified 2026

Spectral Phase-Lifted Features

Augment a hidden representation with positively homogeneous interaction features built from approximate eigenmodes of a linear layer. Fractional products of mode magnitudes and phases provide nonlinear channels whose transformation laws are inherited from the spectrum of the underlying operator, potentially representing oscillatory or multiplicative dynamics more compactly than a generic MLP.

Useful5/10
Difficulty7/10
Novelty8/10
Paper: The spectrum of operator extensions to free Banach Lattices arXiv:2608.11437
Unverified 2026

Independent-Set Neural Output Head

Replace an unconstrained categorical or multilabel output head with a graph-supported distribution over feasible independent sets. Given neural logits, assign probability proportional to the exponential of the total logit of each selected vertex, so incompatible vertices can never be jointly active. Use exact junction-tree inference for decomposable graphs with small treewidth, and compare against post-hoc masking or penalty-based constraint enforcement.

Useful5/10
Difficulty6/10
Novelty6/10
Paper: Graphical Models for Multivariate Count Data arXiv:2608.11366
Unverified 2026

Cross-Order Hodge Stability Scaling

Normalize every higher-order simplicial message-passing or diffusion block using the spectral radius of a lower-order up-Laplacian, rather than estimating a separate radius for each order. The paper's monotonicity theorem guarantees that this shared bound is conservative for all higher orders, enabling stable explicit updates with one spectral calibration.

Useful5/10
Difficulty5/10
Novelty7/10
Paper: Eigenvalue growth of the discrete Hodge Laplacian across dimensions arXiv:2608.11170
Unverified 2026

Elliptic Low-Oscillation Feature Layer

Construct a spatially varying diffusion layer whose coefficient matrix is explicitly uniformly elliptic and whose local mean oscillation is penalized. Use it inside an implicit residual block, so the learned operator remains a controlled perturbation of a constant-coefficient elliptic operator rather than becoming an unstable collection of unrelated local filters.

Useful5/10
Difficulty6/10
Novelty8/10
Paper: Regularity for Elliptic Equations with Coefficients of Small Mean Oscillation arXiv:2608.10813
Unverified 2026

Birkhoff Modular Mask Regularizer

Represent structured neural masks or routing states as order ideals of a finite prerequisite poset, then use a modular score whose exact minimizers are a desired decomposition-closed family of valid configurations. This replaces many pairwise constraint penalties with one additive potential that gives zero cost to every intended valid state and positive cost to invalid intermediate states.

Useful5/10
Difficulty6/10
Novelty7/10
Paper: Decomposition-Closed Sublattices as Minimizer Sets of Modular Functions over Distributive Lattices arXiv:2608.10026
Unverified 2026

Asymmetry-Aware Bochner–Riesz Graph Filter

Replace a sharp graph-Laplacian spectral filter with a Bochner–Riesz filter whose smoothness exponent increases when the graph contains regions with different effective dimensions. Estimate the largest local dimension and dimension gap from neighborhood growth, then choose the exponent above both the classical spectral threshold and the asymmetric obstruction threshold. This should suppress unstable high-frequency mixing in heterogeneous graphs while preserving more low-frequency signal than…

Useful5/10
Difficulty5/10
Novelty6/10
Paper: Herz versus Fefferman: Symmetric and asymmetric Bochner--Riesz theory arXiv:2608.09247
Unverified 2026

Mean-Curvature Relaxation Layer

Insert a small number of differentiable graphical mean-curvature-flow steps between a neural network's raw vector-field prediction and its task loss. The relaxation performs geometry-aware smoothing rather than isotropic Gaussian smoothing, and it can enforce fixed boundary values after every step.

Useful5/10
Difficulty5/10
Novelty7/10
Paper: Well-posedness for the mean curvature flow on the half-space and on bounded domains arXiv:2608.08901
Unverified 2026

Random-cluster anti-correlated routing

Replace independent Bernoulli branch dropout in a tree-structured mixture or hierarchical MLP with connectivity gates sampled from a q<1 wired random-cluster model. The q<1 law provides conditional negative association across branches, so increasing statistics of disjoint branches have nonpositive covariance; this should reduce redundant expert activation while preserving structured stochastic exploration.

Useful5/10
Difficulty6/10
Novelty6/10
Paper: The $q<1$ Random-Cluster Model on Wired Trees: Uniqueness and Negative Dependence arXiv:2608.08565
Unverified 2026

Lexicographic spectral activation

Replace an ordinary elementwise nonlinearity on a learned Hermitian matrix with a matrix function f(A), while supplying exact Jacobian-vector and Hessian-vector products through the lexicographic divided-difference formula. This gives a principled spectral layer for covariance features, graph operators, attention kernels, or matrix-valued embeddings, particularly when perturbation matrices do not commute and eigenvalues are repeated or nearly repeated.

Useful5/10
Difficulty5/10
Novelty5/10
Paper: Lexicographic functional calculus and its application to functional calculus calculus arXiv:2608.08404
Unverified 2026

Seven-Factor Stochastic Transition Layer

Parameterize a learned 3-state transition operator as a product of at most seven elementary row-stochastic matrices rather than learning its nine entries independently. Each factor performs one convex pull-in of row i toward row j, so every intermediate and final matrix remains row-stochastic and the layer has a sparse, bounded-depth interpretation.

Useful5/10
Difficulty4/10
Novelty7/10
Paper: Bang--bang representation of $3\times 3$ embeddable stochastic matrices arXiv:2608.08242
Unverified 2026

Monotone Hardy Mixer

Replace a learned causal mixing profile by a monotone profile followed by a prefix-average Hardy mixer. The monotonicity constraint makes the mixer provably non-degenerate in the BMO sense: localized variation in the profile cannot be reduced below a calibrated factor by prefix averaging. This is a cheap alternative to dense causal attention for tasks where importance or state profiles are expected to decay along sequence position.

Useful5/10
Difficulty4/10
Novelty8/10
Paper: The sharp reverse Hardy inequality in BMO for nonincreasing functions arXiv:2608.08093