Architecture ideas

Research ideas extracted from mathematics papers, categorized as Architecture.

Unverified 2026

Spectral Hamiltonian Neuron

Replace a scalar neuron activation with a matrix function of a learned Hamiltonian. Fixed Hermitian interaction operators are combined as a trainable linear Hamiltonian, the activation is applied to its eigenvalues, and the resulting observable is measured on an input quantum state. Noncommuting interaction terms provide a controlled source of expressivity beyond an ordinary scalar neuron.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Canonical quantization of neurons arXiv:2607.05000
Unverified 2026

Bilinear Input-Conditioned Koopman Cell

Replace an unconstrained input-conditioned recurrent transition with a bilinear latent update, so controls modulate a fixed linear latent dynamics matrix through low-rank state-input interactions. The resulting cell preserves the computational simplicity of linear propagation while representing multiplicative effects of actions that an additive control term cannot capture efficiently.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Koopman operator theory: fundamentals, control, and applications arXiv:2607.01819
Unverified 2026

Bounded Commuting Cochain Layer

Replace independently predicted node, edge, and face features on a simplicial mesh by a coupled projection layer that is idempotent, bounded in a mass-matrix norm, and approximately commutes with the discrete exterior derivative. The layer can be inserted after an ordinary graph-neural update and should suppress topologically inconsistent feature components without requiring the downstream network to learn these constraints from data.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: An Approximate Bounded Cochain Projection arXiv:2607.07457
Unverified 2026

Binary-form symmetric-power equivariant layer

Replace an unconstrained feature vector of size n+1 by the coefficients of a homogeneous degree-n binary polynomial and make the layer transform through the irreducible symmetric-power representation of GL_2(R). For n=4 this creates a five-channel equivariant feature block whose transformation law is exact rather than learned through augmentation.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: On 4-dimensional convex projective domains invariant by a lattice of $\mathrm{SL}_2 (\mathbb{R})$ arXiv:2607.07150
Unverified 2026

Hankel Residual Observer

Attach a model-free residual-dynamics observer to a neural multi-step forecaster. Instead of asking the network to relearn persistent periodic or autoregressive disturbances, maintain a Hankel dictionary of recent forecast errors and use ridge reconstruction to predict the next residual sequence online. Add the predicted residual to the network forecast with a confidence-dependent correction gain.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: Model-Free Disturbance Observer with Online Modification: Listening to MFDOOM arXiv:2607.07082
Unverified 2026

Observability-Gated Spectral Phase Initialization

Add a preprocessing or differentiable synchronization layer that estimates one unit-modulus complex phase per graph node or data view from noisy pairwise relative-phase observations. Initialize the phases with a leading-eigenvector method, fix the global phase gauge, and allow nonlinear refinement only when the estimated perturbation is small relative to the observable Jacobian margin. This replaces random initialization for rotation-alignment modules and should reduce bad local minima caused…

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Spectral Initialization and Certification for Power System Angle Estimation arXiv:2607.06762
Unverified 2026

Compressed threshold-overlap Gram layer

Represent each k-element object by a vector in dimension \(r=\binom{n-2(k-s)}{s}\), and use a PSD Gram matrix to encode the rule that pairs with intersection smaller than s have zero similarity while pairs with intersection at least s have nonzero similarity. Insert this representation into set encoders, graph neural networks, or overlap-aware attention instead of allocating one feature for every s-subset.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Minimum-rank parameters of complements of threshold Kneser graphs arXiv:2607.06480
Unverified 2026

Periodic CMV Unitary Recurrent Layer

Replace a dense recurrent transition matrix with a periodic CMV-style product of alternating local 2x2 unitary cores. The transition is exactly norm-preserving, has O(n) trainable parameters under periodic tying, and can be applied through local factor operations rather than stored as an n-by-n matrix. Use turnover refactorization when changing the ordering or boundary connection of cores, enabling a compact cyclic unitary state-space layer.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Fast computation of eigenvalues of periodic CMV matrices arXiv:2607.06400
Unverified 2026

Fractional Mahalanobis radial head

Replace a binary classifier's unconstrained final logit with a differentiable likelihood-ratio head based on two squared Mahalanobis radii in a learned embedding space. Approximate the shared radial generator with a small fractional-power basis, allowing the head to model heavy-tailed class geometry that an affine QDA logit cannot represent while remaining much smaller than a generic nonlinear head.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Closed-form fractional radial links for elliptical Mahalanobis discriminant analysis arXiv:2607.06089
Unverified 2026

Adaptive CUR Neural Layer

Replace a dense weight matrix by a cross approximation built from selected rows and columns rather than by a conventional truncated SVD. Periodically refresh the selected indices using residual leverage scores, warm-starting from the previous factorization so that the compressed layer can track weight changes during fine-tuning.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: A Sub-linear Low-Rank Solver for Poisson's Equation using Machine Learning Frameworks for GPU Acceleration arXiv:2607.06021
Unverified 2026

Dual-unitary recurrent state block

Replace a generic recurrent transition with two coupled unitary transitions that share one block column and differ by a sign on the other block column. Each transition preserves hidden-state norm exactly, while the structured difference gives a controlled two-path recurrent architecture for long-context modeling.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Operator model and a trace formula for pairs of unitary operators arXiv:2607.05334
Unverified 2026

Pole-Certified SSM Initialization

Extract a small set of stable exponential modes from an observed neural sequence and use them to initialize a diagonal or block-diagonal state-space model. Hankel-pencil eigenvalues propose the modes, while persistence across shifts and contour margins reject modes caused by noise or a short-lived background.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Determinant Characteristics and Argument-Principle Certification for Visible Poles in Meromorphic Continuation arXiv:2607.04568
Unverified 2026

Unassembled Adaptive Cell Neural Network

Represent every mesh interface degree of freedom by one feature copy per incident cell, and apply local neural blocks directly to these cell tensors. Enforce inter-cell consistency with valence-weighted averaging only after selected layers or hierarchy transitions, avoiding repeated construction of a global sparse graph or assembled feature vector. This is suited to adaptive quadtrees, octrees, and finite-element neural operators.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Coalesced Matrix-Free Geometric Multigrid on Persistent Cell-Wise Storage arXiv:2607.03413
Unverified 2026

Expander OMP Sparse Inference

Use the expander decoder as a structured sparse-coding dictionary and replace dense OMP correlation steps with edge-wise gather-and-reduce operations. This is useful when codes must be inferred iteratively, including interpretable feature extraction, sparse retrieval, or an inference-time latent selector that cannot rely entirely on an amortized encoder.

Useful6/10
Difficulty4/10
Novelty6/10
Paper: Expander Sparse Autoencoders: Parameter-Efficient Dictionaries for Mechanistic Interpretability arXiv:2607.01799
Unverified 2026

Totally-positive bidiagonal mixer

Replace a dense channel or token-mixing matrix with a product of positive bidiagonal factors, so information propagates through a controlled sequence of local couplings rather than arbitrary signed interactions. Initialize the factors from the paper's barycentric-subdivision factorization, then learn positive diagonal and off-diagonal parameters; the resulting map is structured, parameter-efficient, and constrained to remain totally positive.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Total positivity of transformation matrices for uniform subdivisions arXiv:2607.01577
Unverified 2026

Monotone Singular-Value ICNN Envelope

Replace a generic neural constitutive law or energy model with an ICNN that consumes the positive singular values of a deformation-like matrix and is convex and coordinatewise nondecreasing in those inputs. Train it as a lower approximation to a nonconvex target energy, so the network acts as a computationally cheap sufficient polyconvex-envelope surrogate rather than merely interpolating unstable samples.

Useful6/10
Difficulty4/10
Novelty4/10
Paper: Compression of Polyconvex Envelopes of Isotropic Functions via Monotonic Input Convex Neural Networks arXiv:2607.01055
Mechanism failed 2026

Intrinsic-Capacity Feature Bottleneck

Regularize an intermediate neural representation according to its estimated low-dimensional separability capacity instead of its ambient feature width. Learn feature gates or subspace assignments, estimate the union of active supports, and penalize representations whose Cover capacity exceeds a task-dependent target.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Function-Counting Theory for Low-Dimensional Data Structures arXiv:2607.01010
Unverified 2026

Orthogonal-Rank Contextual Memory

Replace a discrete or one-hot recurrent state table with a low-dimensional vector memory whose event embeddings are orthogonal whenever the corresponding events are mutually exclusive in an input exclusivity graph. The module uses continuous state vectors and can therefore target dimension \(d=\xi(G)\), whereas a discrete state encoding is lower-bounded by \(N\geq\chi(G)\). This should be tested on graph-defined formal-language recognition tasks, where the graph is known and the claimed…

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Quantum Memory Advantage from Contextuality arXiv:2607.00507
Unverified 2026

Independent-Simplex Hypergraph Router

Use the paper's edge-to-area incidence structure to choose a small set of geometrically independent simplices instead of processing every possible hyperedge. A greedy rank-increasing router retains a triangle only when its Jacobian adds a new direction, reducing higher-order message-passing cost while preserving diverse geometric information.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: On volume vectors determined by hypergraphs in thin subsets of Euclidean space arXiv:2607.00153
Unverified 2026

Jacobian-Ranked Simplex Features

Add a differentiable hypergraph layer that converts invariant edge-length features into triangle areas or higher-dimensional simplex volumes before message passing. Select or weight simplices according to the singular values of the length-to-volume Jacobian, so the network receives geometrically independent features rather than many redundant or nearly degenerate measurements.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: On volume vectors determined by hypergraphs in thin subsets of Euclidean space arXiv:2607.00153
Unverified 2026

Pole-safe rational neural layer

Replace an unconstrained feature vector entering a rational or resolvent-like neural operator by a polynomial feature whose first nonzero Taylor coefficient lies in a pole-safe subspace. For a pole of order m, the simplest guaranteed construction is psi(z)=(z-beta)^m v, which makes Q(z)psi(z) bounded even when Q(z) diverges. For lower-order cancellation, solve linear constraints among Taylor coefficients of psi so that all negative Laurent powers vanish.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Construction of Pole Cancellation Functions at Ordinary Poles of Operator-Valued Functions arXiv:2607.00097
Unverified 2026

GQL Safe Residual Layer

Insert a scalar flux-correction-style limiter after a neural operator predicts a conservative state or residual. Interpolate between a known-admissible baseline state and the learned high-order candidate, choosing the largest coefficient that satisfies a geometric family of linear inequalities encoding positive density, positive pressure, and subluminal velocity. This retains as much of the neural prediction as possible instead of independently clipping physical variables.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: GQL-Based Physical-Constraint-Preserving High-Order Finite Difference Schemes for Special Relativistic Hydrodynamics in Arbitrary Dimensions arXiv:2606.31992
Unverified 2026

Path-Holonomy Attention

Replace ordinary additive path aggregation in graph attention with ordered products of edge operators equipped with learned reversal and color-switch maps. Closed-loop products become a consistency signal, allowing the model to retain direction-sensitive relational information that standard permutation-invariant message passing can lose.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Noncommutative Cluster Varieties and Moduli Spaces of Local Systems arXiv:2608.27284
Unverified 2026

Reversible Matrix Cluster Layer

Construct a latent layer whose node states are small positive-definite matrices and whose local updates follow a weighted cluster exchange relation rather than an unconstrained affine map. The update is reversible when the old state is retained, while noncommuting matrix products preserve relational structure that scalar cluster variables cannot represent.

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Noncommutative Cluster Varieties and Moduli Spaces of Local Systems arXiv:2608.27284