Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

Unverified 2026

Low-Rank Gromov-Wasserstein Cross-Attention

Replace ordinary feature-only cross-attention between two graph sets with a transport coupling optimized using pairwise structural costs. Use the paper's positive-semidefinite graph-kernel factorization to compute the quadratic GW interaction without materializing an O(N_0^2 N_1^2) tensor, and use the inexact-gradient convergence bound to control truncated or approximate updates. The resulting coupling can be used directly as attention weights or as a soft graph-matching matrix.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Discrete Gromov-Wasserstein Duality: Algorithms and Isomorphism Testing arXiv:2609.03094
Unverified 2026

Symplectic Polar Transition Normalization

Normalize a learned symplectic recurrent transition using the paper's explicit positive-definite factor instead of projecting the entire matrix onto an orthogonal group. Retain the compact unitary dynamics as the stable transport component and separately damp only the noncompact factor, providing a tunable stability mechanism that preserves symplectic structure.

Useful6/10
Difficulty6/10
Novelty5/10
Paper: Maslov Indicies In Symplectic Geometry Revisited arXiv:2609.03061
Unverified 2026

Exact hierarchical polar differential layer

Replace a Cartesian grid or singular polar convolution near the center of a disk with a hierarchy of polar-spline coefficient features. Use separate coefficient spaces for scalar fields, vector 1-forms, and 2-forms, together with sparse derivative matrices satisfying the discrete complex identity D1 D0 = 0. Refine only cells with large learned or physics residuals, while retaining the pole constraints that make the representation smooth despite the collapsed angular boundary.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Adaptively-refinable polar-spline discrete differential forms: hierarchical construction, exactness, and applications arXiv:2609.03461
Unverified 2026

Sector-Parametrized Neural Operator

Replace a recurrent, SSM, or residual linear operator W by an exact affine image of a contractive matrix sigma. The resulting W lies in a prescribed matrix ball and, for the positive-real choice of parameters, satisfies a sector inequality such as W+W^* >= 0. This gives an anisotropic, learnable alternative to plain spectral normalization: the allowable operator set is convex and its center and left/right radii can encode known directional scales.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Hyperpositive functions, sector bounded functions and a trace formula arXiv:2609.03403
Unverified 2026

Star-Symmetric Delayed State Layer

Replace an untied stack of recurrent or polynomial-filter blocks with a delayed recurrence whose coefficients repeat with period r and whose characteristic polynomial has exact (r+1)-fold rotational symmetry. The resulting layer couples the current state to a state r steps back, producing a structured spectrum rather than arbitrary eigenvalues and potentially improving long-horizon propagation with fewer parameters.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Ratio and limiting zero distribution asymptotics for symmetric multiple orthogonal polynomials arXiv:2609.02801
Unverified 2026

Minimax-positive spectral graph layer

Replace an arbitrary graph spectral filter with a constrained transition-band filter that is exactly one on the lowest L graph frequencies and zero above frequency N. Train or initialize the transition coefficients to minimize the induced infinity-norm excess above one, which controls amplification and sign overshoot while retaining low-frequency information exactly. The paper predicts that widening the transition band reduces the best achievable excess quadratically rather than linearly.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Positivity loss in bandlimited spectral reproduction on spheres arXiv:2609.02695
Unverified 2026

Offline quadratic latent operator

Replace repeated high-dimensional quadratic interactions in a neural operator or world model with a reduced latent quadratic map whose basis-pair interactions are precomputed offline. The online computation becomes a small polynomial in latent coefficients, preserving quadratic interaction structure while avoiding repeated full-resolution contractions.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Reduced order model for parametric Boltzmann equation and its application to inverse problems arXiv:2609.02578
Unverified 2026

Bicomplex Spatio-Temporal Residual Encoder

Replace independent per-sensor temporal encoders with a shared two-dimensional transform whose temporal and spatial phase channels remain algebraically distinguishable. Train a neural residual detector and localization head on the transformed coefficients, forcing perturbations with the same temporal signature but different positions to differ primarily in their spatial phase.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Multivariable Geometric Laplace Transform and Fault Detection in Distributed-Converter Lines arXiv:2609.02431
Unverified 2026

Quantum-Expander Covariance Mixer

Insert a fixed expander channel before a covariance-dependent feature transformation. The channel repeatedly conjugates the feature covariance by a constant number of sparse Pauli/CNOT unitaries, preserving total feature energy while contracting anisotropic covariance components. Use the mixed covariance for whitening or as a regularized normalization statistic, and test whether it gives more stable training than dense whitening or an explicit isotropy penalty.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Depth-1 expanders on the unitary group and applications arXiv:2609.01605
Unverified 2026

Signed de Bruijn parity memory

Augment an RNN or state-space model with a finite-state binary-context module whose transitions are those of a de Bruijn graph, while a signed transition channel records a quadratic parity function of the recent context. The exact finite-memory branch preserves cancellation-sensitive parity features that a continuous hidden state may forget, and a learned readout can combine it with the ordinary neural state.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: The Quadratic Easy Coefficients Conjecture via Finite-Type Shifts and Zeta Functions arXiv:2609.01399
Unverified 2026

Reciprocal Beta-Angle Mixer

Insert a projection-space mixer that combines several fixed or learned directions using reciprocal correlations with the current feature, then normalize the result. The exact construction has a universal beta law for its squared input-output cosine, so it can create controlled angular diversity while remaining deterministic and independent of the chosen direction dictionary.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Universal Beta Incidence Angles: Cauchy Rigidity and Infinite Arrangements arXiv:2609.00603
Unverified 2026

Polyhedral Column-Generation MoE Router

Replace unconstrained token-to-expert routing with a nonnegative mixture of a small set of feasible routing or communication patterns. Enforce resource limits using the paper's one-sided positive-contribution bound, which gives a conservative certificate without enumerating all joint token activation scenarios. Expand the pattern set only when a separation procedure finds a routing pattern that improves the router objective while adding useful capacity information.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Models and Algorithms for Reserve Deliverability in Cross-Zonal Balancing Capacity Markets arXiv:2609.00439
Unverified 2026

Dual-gauge cross-stream block

Replace an unconstrained hidden-to-hidden interaction in an MLP or transformer feed-forward block by two gauge-related branches. Split channels with an orthogonal involution Θ, constrain the learned interaction K to anticommute with Θ, and use opposite signs of K in paired branches. This creates a testable inductive bias in which the learned interaction only transfers information between the two channel subspaces.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Gauge-compatible tensors on statistical manifolds: splitting and submanifold geometry arXiv:2608.31145
Unverified 2026

Derivative-Free Dynamic-Stiffness PINN

Replace pointwise high-order derivative residuals in an eigenvalue PINN by an assembled dynamic-stiffness residual \(\mathbf W(\omega)q_\theta\), where each element matrix is obtained from homogeneous PDE solutions. The network predicts nodal degrees of freedom or element boundary traces, while the exact frequency-domain operator enforces the physics without differentiating the network multiple times with respect to coordinates.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: A Framework Integrating the Dynamic Stiffness Matrix with Physics-Informed Neural Networks for Solving Eigenvalue Problems and Analysing Dynamic Response arXiv:2608.28683
Unverified 2026

Pressure-Controlled Neural IFS

Construct a generative or recurrent neural architecture with several contractive or mildly expanding branches, and explicitly control the geometric complexity of its invariant set using the sub-additive singular-value pressure of branch-Jacobian products. Instead of regularizing only the operator norm, the model can preserve anisotropic directions while targeting a desired attractor dimension, potentially improving coverage of structured data without uncontrolled folding or collapse.

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Dimensions of surface repellers and attractors of non-linear planar IFSs arXiv:2608.30744
Unverified 2026

PCA-Discovered Implicit Latent Dynamics

Replace an explicit recurrent transition with a learned descriptor relation in latent space, allowing some latent coordinates to satisfy algebraic constraints rather than being numerically integrated. Fit the relation using total-least-squares or iterative PCA on the jointly observed trajectory, so noise in every channel is treated symmetrically and the model can discover whether the latent system is index-0 or index-1.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Subspace Based Identification of Errors-in-Variables Linear Descriptor Systems arXiv:2608.30259
Unverified 2026

Relative-Degree-Gated Passive Neural State Space

Construct a neural state-space model with an explicit first-order input-to-output path instead of forcing every output to depend only on deeply propagated hidden states. Penalize or reject learned linearizations whose transfer matrix has relative degree greater than one, then train a storage-function certificate for the remaining passive dynamics. This preserves the paper's relative-degree compatibility condition while allowing high-order internal memory.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Relative-Degree Wall Restricts Passivity-Based Stability Analysis in Inverter-Dominant Grids arXiv:2608.29474
Unverified 2026

Hard-Saturated Neural Feedback

Build actuator or parameter constraints directly into the neural controller using a differentiable hard-saturation map rather than penalizing violations after the fact. This makes the Lyapunov certificate apply to the actual bounded controller and prevents training from exploiting unrealistically large actions.

Useful6/10
Difficulty3/10
Novelty4/10
Paper: Learning neural controllers for nonlinear systems from data arXiv:2608.29303
Unverified 2026

Laguerre-Optimal Positive Delay Filter

Replace an Erlang delay or exponential smoothing cascade in a recurrent or state-space layer by a positive rational kernel of the form \(\kappa(u)=C e^{-a u}p(u)^2\). Choose the degree-\(m\) polynomial by deleting the adjacent pair of Laguerre zeros with smallest relative gap from \(L_{m+2}\), then rescale the resulting density to unit mean. This preserves a nonnegative impulse response while reducing temporal jitter relative to Erlang filters.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Least Variability in a Polynomial-Square Class of Rational Kernels arXiv:2608.29143
Unverified 2026

Spectral Sign-Balanced Update Blocks

Represent a block of candidate neural updates or adapter components by symmetric influence matrices and select one sign for each component so their aggregate spectral effect is small. This imports matrix discrepancy into low-rank adapters, expert aggregation, or structured quantization, where controlling the worst direction of interference may be more useful than minimizing entrywise error.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: A Proof of the Matrix Spencer Conjecture arXiv:2608.28816
Unverified 2026

Minor-preserving complex network layer

Parameterize a complex linear layer as a product of sparse triangular network factors whose positive modulus version is totally nonnegative. The layer can use phase cancellation for expressive transformations, while selected minors remain bounded by explicitly computable positive minors, giving a structured alternative to unconstrained dense complex weights.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Entropy and domination for quasi-Hitchin representations arXiv:2608.27939
Unverified 2026

Concatenated-Neighbor Low-Rank Message Passing

Replace a sparse graph layer's separate edge transformations with one joint low-rank factorization of all transformations entering each target node. For target node i, concatenate the neighbor matrices horizontally, project all neighbor features into a shared low-dimensional receiving basis, and reconstruct one output; retain the self transformation exactly. This can reduce edge-parameter storage and message-passing FLOPs when the incoming block row has rapidly decaying singular values.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Accelerated S-NFC for Million-Chaff RCS Computation Using Low-Rank Compression of Concatenated Block Rows arXiv:2608.27936
Unverified 2026

Cost-Map Finite-Action Head

Replace online enumeration over a finite action set with a classifier or lookup map whose regions directly return the action minimizing a one-step predictive-control cost. For affine dynamics and quadratic tracking loss, exact action regions are separated by pairwise cost boundaries, so the approximation can be audited against exhaustive predictive control rather than treated as an unconstrained policy.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: A Simple and Extremely Efficient Predictive Control for Power Converters arXiv:2608.22416
Unverified 2026

Indefinite Sketched Nyström Interaction

Approximate a dense symmetric interaction matrix in a neural layer by \(\widehat A=C\widehat M C^{\top}\), but compute the small core \(\widehat M\) from a two-sided sketched least-squares fit rather than from the landmark principal submatrix. This preserves signed or indefinite directions and avoids exploding outputs caused by an almost-singular \(A(I,I)\).

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Nyström method for symmetric indefinite matrices arXiv:2608.20531