Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

Unverified 2026

Manifold-Adaptive Positive Quadrature Attention

Replace dense aggregation over patches, points, or graph neighbors with a state-dependent positive quadrature rule. A latent encoder produces the current manifold coordinate z, a small controller chooses a subset of entities, and a convex redistribution step assigns nonnegative weights so that the subset preserves the full aggregation on a learned feature basis. The method is especially suitable for neural operators and vision attention where many spatial entities are redundant along the data…

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Dimensional hyperreduction of nonlinear finite element models via empirical cubature with manifold-adaptive weights arXiv:2609.03068
Unverified 2026

Exact hierarchical polar differential layer

Replace a Cartesian grid or singular polar convolution near the center of a disk with a hierarchy of polar-spline coefficient features. Use separate coefficient spaces for scalar fields, vector 1-forms, and 2-forms, together with sparse derivative matrices satisfying the discrete complex identity D1 D0 = 0. Refine only cells with large learned or physics residuals, while retaining the pole constraints that make the representation smooth despite the collapsed angular boundary.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Adaptively-refinable polar-spline discrete differential forms: hierarchical construction, exactness, and applications arXiv:2609.03461
Unverified 2026

Resolution-Invariant Hilbert-Space CNF Prior

Replace a finite-dimensional latent prior for fields or trajectories with a continuous flow acting on function coefficients in a separable Hilbert space. A Gaussian reference with an explicit covariance spectrum is transformed by a learned vector field, so the model can represent smooth, non-isotropic function distributions while sharing parameters across discretization resolutions.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Learning Informative Prior with Infinite-Dimensional Continuous Normalizing Flow for Bayesian Inverse Problem arXiv:2609.03343
Unverified 2026

Copula Correction Layer for Joint Predictive Calibration

Add a post-processing or differentiable output layer that first matches each predicted marginal distribution to the data marginal and then corrects the remaining joint dependence by transporting the predicted copula to the observed copula. This directly targets the discrepancy that sequential marginal or quantity-of-interest constraints cannot remove.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Copula Transformations for Data-Consistent Inversion arXiv:2609.02832
Unverified 2026

Offline quadratic latent operator

Replace repeated high-dimensional quadratic interactions in a neural operator or world model with a reduced latent quadratic map whose basis-pair interactions are precomputed offline. The online computation becomes a small polynomial in latent coefficients, preserving quadratic interaction structure while avoiding repeated full-resolution contractions.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Reduced order model for parametric Boltzmann equation and its application to inverse problems arXiv:2609.02578
Unverified 2026

Bicomplex Spatio-Temporal Residual Encoder

Replace independent per-sensor temporal encoders with a shared two-dimensional transform whose temporal and spatial phase channels remain algebraically distinguishable. Train a neural residual detector and localization head on the transformed coefficients, forcing perturbations with the same temporal signature but different positions to differ primarily in their spatial phase.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Multivariable Geometric Laplace Transform and Fault Detection in Distributed-Converter Lines arXiv:2609.02431
Unverified 2026

Asymmetric Conservative Interaction Layer

Build a neural interaction layer for multiple populations whose cross-population kernels are ordered, so the response of population s to population t need not equal the response of t to s. Evaluate the interaction in divergence form and apply an explicit moment-nullspace projection so each layer preserves total mass, momentum, and energy instead of learning these constraints from penalties.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Learning a general class of admissible multi-species collision operators from molecular dynamics arXiv:2609.01845
Unverified 2026

Sparse Legendre Shape Hypernetwork

Use a sparse multivariate Legendre expansion as the geometry-to-network-weights map, rather than an unconstrained MLP that consumes all shape parameters. The hypernetwork predicts only coefficients for a selected set of polynomial multi-indices, allowing high-dimensional or countably parameterized shape uncertainty to be handled with a number of learned terms determined by coefficient decay rather than ambient dimension.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Shape Holomorphy and Sparse Approximation of the Maxwell Electric Field Integral Operator arXiv:2609.00466
Unverified 2026

Bernstein-Safe Policy Head

Replace a neural controller's pointwise action outputs over a finite horizon with Bernstein control points whose convex hull satisfies actuator and trajectory constraints. The network predicts the control points, while a robust margin accounts for bounded tracking or model-prediction error, making continuous-time actuator feasibility checkable from finitely many inequalities.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Actuator-Aware Spatiotemporal Tube Synthesis for Temporal Reach-Avoid-Stay Tasks arXiv:2607.23040
Unverified 2026

PCA-Discovered Implicit Latent Dynamics

Replace an explicit recurrent transition with a learned descriptor relation in latent space, allowing some latent coordinates to satisfy algebraic constraints rather than being numerically integrated. Fit the relation using total-least-squares or iterative PCA on the jointly observed trajectory, so noise in every channel is treated symmetrically and the model can discover whether the latent system is index-0 or index-1.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Subspace Based Identification of Errors-in-Variables Linear Descriptor Systems arXiv:2608.30259
Unverified 2026

Cost-Map Finite-Action Head

Replace online enumeration over a finite action set with a classifier or lookup map whose regions directly return the action minimizing a one-step predictive-control cost. For affine dynamics and quadratic tracking loss, exact action regions are separated by pairwise cost boundaries, so the approximation can be audited against exhaustive predictive control rather than treated as an unconstrained policy.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: A Simple and Extremely Efficient Predictive Control for Power Converters arXiv:2608.22416
Unverified 2026

Integrable Elliptic Memory Cell

Replace an unconstrained recurrent transition with block-diagonal planar rotations whose angles are learned or conditioned on a slowly varying context variable. The resulting hidden-state norm and each two-dimensional block energy are exactly invariant in the ideal recurrence, preventing exploding or vanishing recurrent dynamics while retaining phase information over long horizons.

Useful6/10
Difficulty4/10
Novelty4/10
Paper: Outer Contact Billiards arXiv:2608.19393
Unverified 2026

Dimension-calibrated contractive branch attractor

Construct a recurrent or generative network from finitely many contractive branches whose hidden-state attractor has a prescribed similarity dimension. The branch contraction ratios determine the target complexity through the equation sum_i r_i^s = 1, while a separation penalty approximates the open set condition and prevents branch collapse or excessive overlap.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: The Moran--Hutchinson formula in semimetric spaces arXiv:2608.16817
Unverified 2026

Surrogate-guided topology search for nonlinear reservoirs

Search sparse reservoir wiring in graph space rather than repeatedly testing every candidate with its full nonlinear dynamics. Use graph descriptors to predict validation accuracy and nonlinear feature selectivity, then spend exact simulations on candidates with high predicted performance or high surrogate uncertainty.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Graph-theoretic design of lasing networks for physical vision arXiv:2608.13097
Unverified 2026

Resonance-Gated Triadic Fourier Layer

Replace unconstrained spectral mixing with a three-component triadic interaction whose strength is determined by the quadratic phase mismatch R(xi,xi_1). Near-resonant products receive high weight because their phases remain coherent, while strongly nonresonant products are attenuated. The resonance bandwidth can be fixed from the frequency grid or learned as a positive parameter.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: On solitary wave solutions with two-frequency parameters to the three-component system of quadratic nonlinear Schrödinger equations arXiv:2608.12983
Unverified 2026

Positive Reflected Bellman Layer

Replace an unconstrained spatial aggregation in a neural PDE surrogate or controlled-dynamics model with a fixed-branch expectation layer. Each output is a maximum over controls of a nonnegative weighted average of next-state values, with reflected overshoots attenuated by Robin factors. Increasing any input value therefore cannot decrease the output, giving a hard monotonicity and positivity property instead of relying on a penalty.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: A Positivity-Preserving Expectation Scheme for Hamilton--Jacobi--Bellman Equations with Oblique Robin Boundary Conditions arXiv:2608.11936
Unverified 2026

Finite-Difference Conditional Neural Operator

Replace a tensor-product network over a low-dimensional state and a large distribution embedding with a neural operator that consumes the distribution vector once and outputs values on a finite-difference grid in the low-dimensional state. Train it with the governing PDE residual, explicit boundary residuals, and optional signed shape constraints, allowing the network to preserve numerical structure that a generic MLP would learn only implicitly.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Mastering Stochastic OLG Models in Continuous Time arXiv:2608.11134
Unverified 2026

Measure-Lifted Entropy Amplifier

Replace a deterministic latent state with a probability measure over latent states, represented by particles or weighted prototypes. Apply the learned latent transition to every particle, so one base trajectory map induces a dynamics on distributions; use an entropy-preservation or entropy-growth regularizer to prevent collapse of the ensemble. The mechanism predicts that any positive base-state trajectory entropy can generate unbounded distinguishability in the ideal measure space through…

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Entropies of compact subsets and supported measures arXiv:2608.09702
Unverified 2026

Nielsen Quaternionic Hyperbolic Latent Layer

Represent each recurrent latent state as a pair of unit quaternions \((q_1,q_2)\in\mathrm{SU}(2)^2\), and evolve it with a composition of elementary Nielsen maps corresponding to a chosen hyperbolic matrix \(A\in\mathrm{SL}(2,\mathbb{Z})\). The layer exactly preserves the group manifold and Haar volume, preserves the commuting locus \(q_1q_2=q_2q_1\), and reproduces toral hyperbolic dynamics there, giving a structured long-horizon prior instead of an unconstrained matrix recurrence.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Quaternionic Extensions of Hyperbolic Toral Automorphisms arXiv:2608.08252
Unverified 2026

Wasserstein-Gated Cached Equilibrium Adapter

Add a small constrained equilibrium layer whose response depends on the current neural state and recent exogenous history, then cache responses keyed by a learned history embedding. For a new history, reuse a cached response only when an empirical Wasserstein distance to the cached history is below a threshold; otherwise run a few inner optimization iterations. The paper's local Holder and trajectory-stability results motivate graceful degradation rather than catastrophic errors for nearby…

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Stability of Differential Stochastic Variational Inequalities with History-Dependent Responses and Transfer Learning arXiv:2608.06923
Unverified 2026

Random-Batch Interaction Layer

Replace an all-pairs interaction or attention-like message field with an unbiased estimator obtained from a uniformly random partition into batches. Keep the partition fixed for \(\tau\) integration steps, then redraw it; the paper's error law predicts that too-long reuse causes a quadratic-in-\(\tau\) error accumulation, while finite-population error decreases as \(N^{-1}\).

Useful6/10
Difficulty4/10
Novelty5/10
Paper: Mean field error estimate of the random batch method for vortex blob dynamics for the 2D Navier--Stokes Equation arXiv:2608.06533
Unverified 2026

Invertible Fourier Surrogate for Periodic Sequence Modeling

Represent periodic input-output behavior using a compact real vector of Fourier coefficients and learn an invertible neural map from input coefficients to output coefficients. Inference then obtains the input representation for a desired periodic output by a single inverse pass instead of iterative optimization through a nonlinear forward model, while the Fourier representation reduces sequence dimensionality when high-rate signals are spectrally sparse.

Useful6/10
Difficulty6/10
Novelty4/10
Paper: Certified Feedforward Tracking for Unknown Nonlinear Systems via Invertible Neural Networks arXiv:2608.06419
Unverified 2026

Contour-reduced parametric SSM

Replace repeated full-dimensional matrix-exponential or ODE solves in a conditioned continuous-time state-space layer with contour quadrature evaluated in a projection basis. The same reduced basis and contour nodes can serve many conditioning vectors, while shifted reduced resolvents provide a stable and differentiable approximation over a prescribed time window.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Contour integral methods and model order reduction for parametric linear control systems arXiv:2608.05363
Unverified 2026

KAM-Stabilized Quasiperiodic Recurrent Memory

Construct a recurrent module with a phase variable and a transverse memory coordinate modeled on a perturbed twist map. Train the transverse state to lie on an invariant graph over the phase, while the phase follows an approximately irrational rigid rotation. A KAM-inspired graph correction and residual penalty should reduce long-horizon drift in recurrent prediction.

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Persistence of invariant graphs for twist maps under analytic perturbations arXiv:2608.05239