Architecture ideas

Research ideas extracted from mathematics papers, categorized as Architecture.

Unverified 2026

Power-Law Volterra Memory

Add a causal memory branch whose weights are generated by the paper's power-type Volterra kernel rather than learned independently at every lag. Learn or softly constrain the exponents so the model can select rough short-memory behavior or smoother long-memory behavior while using only a few parameters. The branch can be implemented as a truncated causal convolution, a multiresolution approximation, or a recurrent state-space realization.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Small ball probabilities and Chung's law of the iterated logarithm for Gaussian Volterra processes with power-type kernels arXiv:2608.05679
Unverified 2026

Braess-aware graph rewiring

Use Kemeny’s constant as a diffusion-quality gate when adding shortcut edges or cliques to a graph used by a GNN. Candidate augmentations are accepted only when they reduce estimated average hitting time, preventing rewiring operations that superficially shorten paths but make the random walk mix more slowly.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Kemeny's constant and Braess cliques in graphs arXiv:2608.04150
Unverified 2026

Contour-reduced parametric SSM

Replace repeated full-dimensional matrix-exponential or ODE solves in a conditioned continuous-time state-space layer with contour quadrature evaluated in a projection basis. The same reduced basis and contour nodes can serve many conditioning vectors, while shifted reduced resolvents provide a stable and differentiable approximation over a prescribed time window.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Contour integral methods and model order reduction for parametric linear control systems arXiv:2608.05363
Unverified 2026

Hitting-Time Adaptive Transformer Depth

Use attention-graph hitting times to identify tokens whose information has not mixed through the network, then route only those tokens through additional Transformer blocks. Tokens with fast reachability exit early, while slow or isolated tokens receive more computation.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Identifying slow relaxation in many-body quantum systems through state-graph geometry and state-graph heterogeneity arXiv:2608.05298
Unverified 2026

KAM-Stabilized Quasiperiodic Recurrent Memory

Construct a recurrent module with a phase variable and a transverse memory coordinate modeled on a perturbed twist map. Train the transverse state to lie on an invariant graph over the phase, while the phase follows an approximately irrational rigid rotation. A KAM-inspired graph correction and residual penalty should reduce long-horizon drift in recurrent prediction.

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Persistence of invariant graphs for twist maps under analytic perturbations arXiv:2608.05239
Unverified 2026

Chain-Current Latent States

Replace an unconstrained geometric latent vector with a state consisting of discrete chain coefficients, a continuous current, and an integral-current curvature. Neural updates are projected through the differential-homology boundary operator, so learned states remain compatible with conservation and boundary structure on meshes or point clouds.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Differential Homology arXiv:2608.05048
Unverified 2026

Automaton-Gated Objective Switching

Augment a neural policy with deterministic DFA states for the task objective and safety constraint, then select among objective-specific policy heads using those states. Before either target is reached, execute a mixed policy; after one target is reached, switch permanently to the policy specialized for the remaining target.

Useful6/10
Difficulty4/10
Novelty5/10
Paper: Optimal Constrained sc-LTL Planning in MDPs via Switching Policies arXiv:2608.05021
Unverified 2026

Spectrahedral Obedience Layer

Insert a differentiable equilibrium layer between a neural payoff/state encoder and the final action recommendations. The layer parameterizes a joint recommendation object and enforces all unilateral-deviation inequalities as positive-semidefinite constraints, preventing the network from producing recommendations that agents have a strict incentive to disobey. A quantum-inspired density-matrix parameterization can model correlated recommendations using PSD matrices rather than factorized action…

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Quantum Bayes Correlated Equilibrium and the Comparison of Quantum Information Structures in Games arXiv:2608.04973
Unverified 2026

PSD-Safe Bernstein Distance Kernel

Replace an unconstrained learnable distance-bias function in a graph neural network or distance-aware attention layer by a Bernstein approximation of a positive-definite circular kernel. The resulting kernel is a degree-n polynomial in normalized distance while preserving positive semidefiniteness of every finite Gram matrix on the circle, preventing training from producing an invalid covariance-like similarity structure.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: Preservation of Positive-Definiteness by Bernstein Operators on the Circle arXiv:2608.04836
Unverified 2026

Sparse Weighing Mixer

Use enumerated weighing matrices as sparse orthogonal channel-mixing operators inside MLPs or residual blocks. Their ternary entries reduce multiplication to signed additions, while exact orthogonality prevents amplification or attenuation of feature norms; a trainable fixed-support version can recover expressivity without giving up computational sparsity.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Constructing Large Orthogonal Minimally Aliased Response Surface Designs Through Enumeration and Combination of Weighing Designs arXiv:2608.04814
Unverified 2026

Orientation-doubling positional channel

For local structures with a forward/reverse ambiguity, expose both ordered directions and add one explicit orientation bit. This creates a shared bidirectional positional encoder that can distinguish reflected neighborhoods without maintaining two completely independent directional encoders.

Useful6/10
Difficulty4/10
Novelty8/10
Paper: Information and Locality in Cayley Graphs arXiv:2608.04608
Unverified 2026

Hall-Certified Sparse Attention Masks

Use the support graph of a sparse attention or routing mask as a feasibility object before applying normalization. Reject or minimally repair masks whose neighborhoods violate Hall's condition, ensuring that every selected query/key subset has enough reachable mass to support the desired marginals.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Sets that Support a Joint Distribution arXiv:2608.04135
Unverified 2026

Geometric Binary Gate Solver

Replace exhaustive optimization of N binary gates by the geometrically admissible sign patterns induced by projections onto a common direction. For two-dimensional gate vectors, enumerate angular cells exactly; for higher-dimensional vectors, sample directions and evaluate only the induced configurations.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Geometry-Informed Optimization of Binary RIS Configurations for Communication and Sensing arXiv:2608.04133
Unverified 2026

Connectivity-certified matching pruning

Represent a neural architecture as an undirected support graph and prune a matching of edges, so no two simultaneously removed edges share an endpoint. Use the paper's degree/connectivity theorem as a certificate that the pruned architecture retains k-edge-connectivity, preserving multiple routes between computational units instead of relying on unconstrained magnitude pruning.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Removable trees and matchings in $k$-connected and $k$-edge-connected graphs arXiv:2608.03643
Unverified 2026

Boundary-State Renormalization GNN

Replace repeated fine-scale message passing on a recursively constructed graph by a finite-state compositional module that summarizes each cell through boundary connectivity states. Child summaries are combined only through admissible configurations, so the network can represent globally consistent paths and forests without enumerating full walks. The same update map is shared across all hierarchy levels, giving logarithmic-depth computation in the graph diameter and a strong inductive bias for…

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Connective Constants on Nested Fractal Graphs arXiv:2608.03497
Unverified 2026

Aligned-Orthogonal Tensor Layer

Represent a neural-network weight tensor by rank-one terms whose mode factors are selected from shared orthonormal bases, and impose the same basis alignment across tensor flattenings. During or after training, retain the largest coefficients to obtain a structured truncation analogous to truncated SVD. This should produce better-conditioned tensorized layers than unconstrained CP factors while preserving a directly controllable accuracy/compute tradeoff.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Unifying singular value decompositions of tensors via aligned orthogonality arXiv:2608.03202
Unverified 2026

Curved-Critical Residual Lattice

Construct a 2D recurrent or residual neural lattice with slowly varying local couplings, while parameterizing those couplings so that an anisotropy invariant remains constant across all spatial and depth locations. The network obtains controlled local propagation velocities rather than arbitrary inhomogeneous amplification, enabling depth-dependent receptive fields while preserving near-critical signal propagation.

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Two dimensional inhomogeneous classical systems at criticality arXiv:2608.02903
Unverified 2026

Diamond-Consistent Two-Route Layer

Construct a neural layer with two independently ordered transformations and train its operators to satisfy the paper's diamond equations, so that applying direction 1 then direction 2 gives the same result as direction 2 then direction 1. Unlike ordinary weight sharing, the mixed identity permits noncommuting operators whose interaction defects cancel exactly.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Non-Abelian Hirota-Miwa Equations for the KPZ Universality Class arXiv:2608.02772
Unverified 2026

Two-chart refinement cell

Build a recurrent ReLU module that evaluates a binary refinement cascade using a fixed-dimensional state and shared cell weights. Replace hard binary digit selection with two overlapping circle coordinates; switch between their affine state updates only at points where the two candidate updates agree, so the switch is an exact continuous piecewise-linear ReLU operation rather than a multiplicative gate.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Neural network realization of binary refinement iterates via a two-chart atlas selector arXiv:2608.02624
Unverified 2026

Tangential FFN Residuals

Project each FFN residual update onto the tangent space of the current token residual direction before adding it to the stream. This preserves the component that changes representation direction while suppressing norm-only motion, which may reduce residual-norm drift and aggregation-induced representation collapse.

Useful6/10
Difficulty4/10
Novelty6/10
Paper: Feed-Forward Steering in Transformer Residual Dynamics arXiv:2608.02071
Unverified 2026

Noncommutative controllability regularizer

Equip a recurrent or state-space layer with multiple noncommuting transition operators and regularize the span of finite operator words applied to the input injection matrix. This discourages hidden directions that cannot be reached from the input and may improve long-range input influence, gradient propagation, and robustness under operator switching.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: A noncommunicative Kalman condition for null controllability of backward stochastic parabolic systems arXiv:2608.01836
Unverified 2026

Fixed/Anti-Fixed Signed Message Passing

Replace a conventional signed-graph message-passing layer with two coupled feature channels: a fixed channel invariant under switching and an anti-fixed channel that changes sign under switching. Unsigned aggregation updates invariant features, while signed aggregation updates anti-invariant features, implementing the paper's sphere-plus-involution representation at the hidden-state level.

Useful6/10
Difficulty4/10
Novelty5/10
Paper: A Lovász Theta Parameter and Theta Body for Signed Graphs arXiv:2608.01716
Unverified 2026

Random-start learned AMP

Construct an unrolled phase-retrieval network that begins with an isotropic Gaussian estimate rather than a spectral initializer. Retain the AMP residual correction and Onsager subtraction, but learn the scalar measurement denoisers and step sizes; use several random starts and select the iterate with the lowest measurement residual.

Useful6/10
Difficulty5/10
Novelty4/10
Paper: Approximate Message Passing with Random Initialization for Phase Retrieval arXiv:2608.01654
Unverified 2026

Exact Involutive Sprugnoli Mixer

Replace a learned sequence-mixing matrix with a structured lower-triangular Sprugnoli operator whose square is exactly the identity. Applying the same operator in reverse reconstructs activations exactly, so it can be used as a reversible Transformer mixer or reversible channel permutation while retaining nontrivial long-range mixing.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Square roots in the Appell group and Sprugnoli arrays arXiv:2608.01497