Architecture ideas

Research ideas extracted from mathematics papers, categorized as Architecture.

Unverified 2026

Unassembled Adaptive Cell Neural Network

Represent every mesh interface degree of freedom by one feature copy per incident cell, and apply local neural blocks directly to these cell tensors. Enforce inter-cell consistency with valence-weighted averaging only after selected layers or hierarchy transitions, avoiding repeated construction of a global sparse graph or assembled feature vector. This is suited to adaptive quadtrees, octrees, and finite-element neural operators.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Coalesced Matrix-Free Geometric Multigrid on Persistent Cell-Wise Storage arXiv:2607.03413
Unverified 2026

Kolmogorov-Lie Unitary Layer

Build an input-conditioned unitary transformation as an ordered product of exponentials of anti-Hermitian matrices, with each factor controlled by a univariate function of one input coordinate or one learned scalar projection. This replaces a dense multivariate matrix-valued controller with separable scalar nonlinearities while preserving exact unitarity at every forward pass.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Quantum Kolmogorov--Arnold representation theorem for continuous unitary-valued maps arXiv:2607.03187
Unverified 2026

Singularity-Enriched Neural Ansatz

Add an explicit local power-law singular basis to a neural field near mixed Dirichlet-Neumann junctions, allowing the neural network to learn only the smoother remainder. Use the predicted or fitted singular exponent to concentrate collocation points near the junction. This directly targets the regularity bottleneck identified by the paper, where increasing polynomial degree or network capacity cannot overcome a convergence cap under uniform resolution.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: A Unified CutFEM Formulation for Finite-Strain Elasticity: Energy Minimisation and Corner Singularities arXiv:2607.02334
Unverified 2026

Fractional Boundary-Factored Neural Solver

For a fractional Dirichlet problem, replace a free coordinate network N_theta(x) with u_theta(x)=d(x)^a N_theta(x), where d(x)=dist(x,boundary) and 0<a<1 is the fractional order. Train the regular quotient v_theta=u_theta/d^a=N_theta and use a weighted gradient loss that reflects the paper's boundary estimate.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: The structure of solution spaces for fractional-order operators, with gradient estimates arXiv:2607.02312
Unverified 2026

Fourier-Calibrated Nonlocal Feature Gradient

Augment a CNN with a nonlocal feature-gradient branch that compares each feature vector with a kernel-weighted neighborhood rather than using only pointwise or local convolutional interactions. Regularize this branch using the paper's Fourier multiplier energy, which penalizes feature oscillations according to the kernel spectrum and approaches an ordinary local-gradient operator as the interaction radius tends to zero.

Useful6/10
Difficulty4/10
Novelty6/10
Paper: A Variational Nonlocal Phase-Field Model for Dynamic Fracture in Elastic Solids arXiv:2607.01881
Unverified 2026

Expander OMP Sparse Inference

Use the expander decoder as a structured sparse-coding dictionary and replace dense OMP correlation steps with edge-wise gather-and-reduce operations. This is useful when codes must be inferred iteratively, including interpretable feature extraction, sparse retrieval, or an inference-time latent selector that cannot rely entirely on an amortized encoder.

Useful6/10
Difficulty4/10
Novelty6/10
Paper: Expander Sparse Autoencoders: Parameter-Efficient Dictionaries for Mechanistic Interpretability arXiv:2607.01799
Unverified 2026

Totally-positive bidiagonal mixer

Replace a dense channel or token-mixing matrix with a product of positive bidiagonal factors, so information propagates through a controlled sequence of local couplings rather than arbitrary signed interactions. Initialize the factors from the paper's barycentric-subdivision factorization, then learn positive diagonal and off-diagonal parameters; the resulting map is structured, parameter-efficient, and constrained to remain totally positive.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Total positivity of transformation matrices for uniform subdivisions arXiv:2607.01577
Unverified 2026

Agnostic Geometry-Prior Mixer

Train an unconstrained branch and a geometry-aware branch in parallel, then learn how much to trust the analytic branch. This preserves the benefit of explicit geometry on correctly specified tasks while allowing the model to ignore a misleading or irrelevant prior.

Useful6/10
Difficulty3/10
Novelty7/10
Paper: Geometry-Aware R-Structured Kolmogorov-Arnold Networks arXiv:2607.01449
Unverified 2026

Monotone Singular-Value ICNN Envelope

Replace a generic neural constitutive law or energy model with an ICNN that consumes the positive singular values of a deformation-like matrix and is convex and coordinatewise nondecreasing in those inputs. Train it as a lower approximation to a nonconvex target energy, so the network acts as a computationally cheap sufficient polyconvex-envelope surrogate rather than merely interpolating unstable samples.

Useful6/10
Difficulty4/10
Novelty4/10
Paper: Compression of Polyconvex Envelopes of Isotropic Functions via Monotonic Input Convex Neural Networks arXiv:2607.01055
Unverified 2026

Orthogonal-Rank Contextual Memory

Replace a discrete or one-hot recurrent state table with a low-dimensional vector memory whose event embeddings are orthogonal whenever the corresponding events are mutually exclusive in an input exclusivity graph. The module uses continuous state vectors and can therefore target dimension \(d=\xi(G)\), whereas a discrete state encoding is lower-bounded by \(N\geq\chi(G)\). This should be tested on graph-defined formal-language recognition tasks, where the graph is known and the claimed…

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Quantum Memory Advantage from Contextuality arXiv:2607.00507
Unverified 2026

Tropical support-restricted PINN

Represent the PINN solution in a restricted polynomial or Taylor basis whose exponent set is supplied by tropical support analysis, instead of asking an MLP to discover the local series structure from scratch. The restriction removes coefficients that cannot occur in the formal solution, reducing trainable degrees of freedom and preventing spurious low-order or singular terms.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Tropical Geometry as a Restricted Architecture for Physics-Informed Neural Networks: Applications in Nonlinear Fluid-Structure Examples arXiv:2607.00237
Unverified 2026

Independent-Simplex Hypergraph Router

Use the paper's edge-to-area incidence structure to choose a small set of geometrically independent simplices instead of processing every possible hyperedge. A greedy rank-increasing router retains a triangle only when its Jacobian adds a new direction, reducing higher-order message-passing cost while preserving diverse geometric information.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: On volume vectors determined by hypergraphs in thin subsets of Euclidean space arXiv:2607.00153
Unverified 2026

Jacobian-Ranked Simplex Features

Add a differentiable hypergraph layer that converts invariant edge-length features into triangle areas or higher-dimensional simplex volumes before message passing. Select or weight simplices according to the singular values of the length-to-volume Jacobian, so the network receives geometrically independent features rather than many redundant or nearly degenerate measurements.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: On volume vectors determined by hypergraphs in thin subsets of Euclidean space arXiv:2607.00153
Unverified 2026

Pole-safe rational neural layer

Replace an unconstrained feature vector entering a rational or resolvent-like neural operator by a polynomial feature whose first nonzero Taylor coefficient lies in a pole-safe subspace. For a pole of order m, the simplest guaranteed construction is psi(z)=(z-beta)^m v, which makes Q(z)psi(z) bounded even when Q(z) diverges. For lower-order cancellation, solve linear constraints among Taylor coefficients of psi so that all negative Laurent powers vanish.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Construction of Pole Cancellation Functions at Ordinary Poles of Operator-Valued Functions arXiv:2607.00097
Unverified 2026

GQL Safe Residual Layer

Insert a scalar flux-correction-style limiter after a neural operator predicts a conservative state or residual. Interpolate between a known-admissible baseline state and the learned high-order candidate, choosing the largest coefficient that satisfies a geometric family of linear inequalities encoding positive density, positive pressure, and subluminal velocity. This retains as much of the neural prediction as possible instead of independently clipping physical variables.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: GQL-Based Physical-Constraint-Preserving High-Order Finite Difference Schemes for Special Relativistic Hydrodynamics in Arbitrary Dimensions arXiv:2606.31992
Unverified 2026

Collision-Aware Subset Attention

Replace ordinary one-token-to-one-expert or one-token-to-one-attention routing with a local latent subset router: a pooled observation can be explained by a compatible subset of tokens. Pairwise compatibility scores assign probability to subsets, and each token receives the marginal probability that it belongs to the selected subset. This should help when tokens represent overlapping objects, occluded entities, or multiple features that should be processed jointly.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Gaussian Belief Propagation for Tracking With Unresolved Measurements arXiv:2606.31716
Unverified 2026

Positive Mellin Mixture Gate

Replace a free-form order-dependent gate with a positive mixture of Mellin powers $(1+s)^{-a}$. This gives a small, interpretable module whose response across the order variable is automatically generated by a positive measure and therefore inherits complete monotonicity, log-convexity, and Hankel-moment structure.

Useful6/10
Difficulty4/10
Novelty8/10
Paper: Order-Moment Transport and Hankel Determinants in Special-Function Inequalities arXiv:2606.31647
Unverified 2026

Fast-Class MoE Router

Replace a flat stochastic MoE router by a two-level continuous-time routing model: experts within a group communicate rapidly, while transitions between groups occur slowly. Use the effective class-level stationary distribution as a soft load-balancing prior, reducing routing oscillation while preserving expert specialization.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: The Invariant Measure of Multiscale Markov Chains via Fast Arborescence Factorization arXiv:2606.31596
Unverified 2026

Completely Monotone Multiscale Attention Decay

Parameterize a relative-position or lag-decay function as a finite positive mixture of exponentials instead of learning arbitrary attention bias values. The resulting kernel is completely monotone on positive distances, so it is nonnegative, decreasing, and has alternating derivative signs; the mixture provides several learned memory scales without allowing oscillatory or unstable long-range biases.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: Bernstein Functions at Work: Coalescents, Copulas, and Subordination arXiv:2607.04467
Unverified 2026

Markov-coded synchronization-robust latent streams

Replace an independently generated discrete latent sequence or redundancy stream with a symmetric two-state Markov source whose transition probability is tuned or learned. Train the downstream transformer to reconstruct the semantic target after random insertions, while using the paper's insertion-capacity expression to select the latent rate and redundancy budget. The representation should preserve information under timing drift, repeated tokens, and inserted distractors better than iid token…

Useful6/10
Difficulty5/10
Novelty7/10
Paper: The Insertion List-Decoding Capacity and an Improved Bound on the Deletion List-Decoding Capacity arXiv:2607.03989
Unverified 2026

Path-Holonomy Attention

Replace ordinary additive path aggregation in graph attention with ordered products of edge operators equipped with learned reversal and color-switch maps. Closed-loop products become a consistency signal, allowing the model to retain direction-sensitive relational information that standard permutation-invariant message passing can lose.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Noncommutative Cluster Varieties and Moduli Spaces of Local Systems arXiv:2608.27284
Unverified 2026

Reversible Matrix Cluster Layer

Construct a latent layer whose node states are small positive-definite matrices and whose local updates follow a weighted cluster exchange relation rather than an unconstrained affine map. The update is reversible when the old state is retained, while noncommuting matrix products preserve relational structure that scalar cluster variables cannot represent.

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Noncommutative Cluster Varieties and Moduli Spaces of Local Systems arXiv:2608.27284
Unverified 2026

Minkowski-Additive Convex Latents

Store a convex object as a direction-indexed vertex tuple and implement composition of objects through componentwise Minkowski addition and nonnegative scaling. This creates a structured residual or compositional layer where convexification is nonexpansive, making perturbation amplification controllable and avoiding repeated generic geometric optimization.

Useful6/10
Difficulty4/10
Novelty8/10
Paper: Galerkin approximations to the space of convex bodies by polytopes in nondegenerate V-representation arXiv:2608.26615
Unverified 2026

Dual-Rail Ratio Arithmetic Layer

Replace scalar neural activations by pairs of nonnegative channels whose ratio represents the signed or unsigned activation. Implement multiplication and addition through pair algebra, and renormalize each pair because the representation is invariant under multiplying both rails by the same positive scalar. This creates an explicitly bounded, cancellation-aware arithmetic layer for deep multiplicative MLPs, rational networks, and neural fields.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Instantaneous arithmetic computation via ratio-encoding in chemical reaction networks arXiv:2608.26305