Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

Unverified 2026

Bernstein Bounded Implicit Representation

Parameterize a scalar or vector implicit neural field on each spatial cell with Bernstein polynomials and constrain its coefficients instead of sampling many points to enforce output bounds. The Bernstein convex-hull property gives a deterministic pointwise bound everywhere in the cell, making the method useful for neural fields representing densities, concentrations, masks, or material parameters.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: Bounds-Constrained Finite Element Approximation of Time-Dependent Partial Differential Equations arXiv:2609.01915
Unverified 2026

Asymmetric Conservative Interaction Layer

Build a neural interaction layer for multiple populations whose cross-population kernels are ordered, so the response of population s to population t need not equal the response of t to s. Evaluate the interaction in divergence form and apply an explicit moment-nullspace projection so each layer preserves total mass, momentum, and energy instead of learning these constraints from penalties.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Learning a general class of admissible multi-species collision operators from molecular dynamics arXiv:2609.01845
Unverified 2026

Quantum-Expander Covariance Mixer

Insert a fixed expander channel before a covariance-dependent feature transformation. The channel repeatedly conjugates the feature covariance by a constant number of sparse Pauli/CNOT unitaries, preserving total feature energy while contracting anisotropic covariance components. Use the mixed covariance for whitening or as a regularized normalization statistic, and test whether it gives more stable training than dense whitening or an explicit isotropy penalty.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Depth-1 expanders on the unitary group and applications arXiv:2609.01605
Unverified 2026

Signed de Bruijn parity memory

Augment an RNN or state-space model with a finite-state binary-context module whose transitions are those of a de Bruijn graph, while a signed transition channel records a quadratic parity function of the recent context. The exact finite-memory branch preserves cancellation-sensitive parity features that a continuous hidden state may forget, and a learned readout can combine it with the ordinary neural state.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: The Quadratic Easy Coefficients Conjecture via Finite-Type Shifts and Zeta Functions arXiv:2609.01399
Unverified 2026

Marginal Fractional Coupling Layer

Replace a local smoothness penalty or local state transition along a sequence or depth coordinate by a marginal fractional quadratic energy with Fourier multiplier |k|. The sigma=1 kernel is nonlocal and scale-free, so it can preserve long-range correlations while suppressing high-frequency instability more selectively than an ordinary Laplacian penalty.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: BKT-like Correlation Scaling and Twist Responses in a One-Dimensional Fractional $U(1)$ Ginzburg--Landau Model arXiv:2609.00721
Unverified 2026

Fractional Ellipsoidal Sparse Attention

Replace dense attention weights with a compactly supported anisotropic bump derived from the obstacle solution, using one learnable ellipsoid per attention head or feature group. Tokens outside the learned ellipsoid receive exactly zero weight, while tokens inside receive smoothly decaying weights according to a fractional exponent. The learned positive-definite matrix represents orientation, scale, and correlations between feature dimensions.

Useful6/10
Difficulty6/10
Novelty5/10
Paper: Ellipsoidal Positivity Sets for Fractional Obstacle Problems with Quadratic Forcing arXiv:2609.00703
Unverified 2026

Reciprocal Beta-Angle Mixer

Insert a projection-space mixer that combines several fixed or learned directions using reciprocal correlations with the current feature, then normalize the result. The exact construction has a universal beta law for its squared input-output cosine, so it can create controlled angular diversity while remaining deterministic and independent of the chosen direction dictionary.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Universal Beta Incidence Angles: Cauchy Rigidity and Infinite Arrangements arXiv:2609.00603
Unverified 2026

Sparse Legendre Shape Hypernetwork

Use a sparse multivariate Legendre expansion as the geometry-to-network-weights map, rather than an unconstrained MLP that consumes all shape parameters. The hypernetwork predicts only coefficients for a selected set of polynomial multi-indices, allowing high-dimensional or countably parameterized shape uncertainty to be handled with a number of learned terms determined by coefficient decay rather than ambient dimension.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Shape Holomorphy and Sparse Approximation of the Maxwell Electric Field Integral Operator arXiv:2609.00466
Unverified 2026

Polyhedral Column-Generation MoE Router

Replace unconstrained token-to-expert routing with a nonnegative mixture of a small set of feasible routing or communication patterns. Enforce resource limits using the paper's one-sided positive-contribution bound, which gives a conservative certificate without enumerating all joint token activation scenarios. Expand the pattern set only when a separation procedure finds a routing pattern that improves the router objective while adding useful capacity information.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Models and Algorithms for Reserve Deliverability in Cross-Zonal Balancing Capacity Markets arXiv:2609.00439
Unverified 2026

Dual-gauge cross-stream block

Replace an unconstrained hidden-to-hidden interaction in an MLP or transformer feed-forward block by two gauge-related branches. Split channels with an orthogonal involution Θ, constrain the learned interaction K to anticommute with Θ, and use opposite signs of K in paired branches. This creates a testable inductive bias in which the learned interaction only transfers information between the two channel subspaces.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Gauge-compatible tensors on statistical manifolds: splitting and submanifold geometry arXiv:2608.31145
Unverified 2026

Derivative-Free Dynamic-Stiffness PINN

Replace pointwise high-order derivative residuals in an eigenvalue PINN by an assembled dynamic-stiffness residual \(\mathbf W(\omega)q_\theta\), where each element matrix is obtained from homogeneous PDE solutions. The network predicts nodal degrees of freedom or element boundary traces, while the exact frequency-domain operator enforces the physics without differentiating the network multiple times with respect to coordinates.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: A Framework Integrating the Dynamic Stiffness Matrix with Physics-Informed Neural Networks for Solving Eigenvalue Problems and Analysing Dynamic Response arXiv:2608.28683
Unverified 2026

Poisson Non-Backtracking Fingerprint

Add a fixed or lightly parameterized graph-level feature extractor based on short non-backtracking cycle counts, and concatenate its Poisson log-likelihood-ratio features with the output of a graph neural network. The feature scaling uses the paper's explicit means, so the network receives statistics that are approximately independent and correctly normalized rather than raw, highly correlated cycle counts.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Analysis of Polynomial Threshold Functions on Random Regular Graphs: Computational Complexity of Detecting Noisy Random Lift arXiv:2608.28539
Unverified 2026

Critical Heavy-Tail Propagation Layer

Insert a sparse heavy-tailed nonlocal mixing operator into a residual sequence, graph, or spatial network so that information can traverse distant positions without stacking many local layers. Use the critical tail exponent s = 1/2, whose truncated first moment grows logarithmically and predicts an effective propagation distance proportional to depth times log depth rather than merely depth.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Propagation rates in integro-differential equations of ignition type arXiv:2608.28450
Unverified 2026

Transverse Quenched Feature Flow

Add a fixed, spatially correlated perturbation field to every layer of a CNN or 2D state-space model, with the perturbation decomposed into transverse and longitudinal Fourier components. Unlike ordinary injected noise, the same field is reused for all training examples and all forward passes, allowing it to act as a structured architectural flow that can promote global feature alignment. Sweep the transverse fraction at fixed total perturbation variance and test for the predicted ordering…

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Correlated disorder versus correlated noise: Ordering in active systems arXiv:2608.28012
Unverified 2026

Bernstein-Safe Policy Head

Replace a neural controller's pointwise action outputs over a finite horizon with Bernstein control points whose convex hull satisfies actuator and trajectory constraints. The network predicts the control points, while a robust margin accounts for bounded tracking or model-prediction error, making continuous-time actuator feasibility checkable from finitely many inequalities.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Actuator-Aware Spatiotemporal Tube Synthesis for Temporal Reach-Avoid-Stay Tasks arXiv:2607.23040
Unverified 2026

Pressure-Controlled Neural IFS

Construct a generative or recurrent neural architecture with several contractive or mildly expanding branches, and explicitly control the geometric complexity of its invariant set using the sub-additive singular-value pressure of branch-Jacobian products. Instead of regularizing only the operator norm, the model can preserve anisotropic directions while targeting a desired attractor dimension, potentially improving coverage of structured data without uncontrolled folding or collapse.

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Dimensions of surface repellers and attractors of non-linear planar IFSs arXiv:2608.30744
Unverified 2026

Trainable two-sided activation gate

Replace the dense output of selected linear projections with a two-sided magnitude threshold that emits zero for small values but preserves signed large values. Learn one positive threshold per projection, or optionally one threshold per output channel, so the network discovers where sparse events can be removed while retaining outlier information.

Useful6/10
Difficulty4/10
Novelty5/10
Paper: Event-Driven Language Models with Sparse Neural Activity for Neuromorphic Hardware arXiv:2608.30439
Unverified 2026

PCA-Discovered Implicit Latent Dynamics

Replace an explicit recurrent transition with a learned descriptor relation in latent space, allowing some latent coordinates to satisfy algebraic constraints rather than being numerically integrated. Fit the relation using total-least-squares or iterative PCA on the jointly observed trajectory, so noise in every channel is treated symmetrically and the model can discover whether the latent system is index-0 or index-1.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Subspace Based Identification of Errors-in-Variables Linear Descriptor Systems arXiv:2608.30259
Unverified 2026

Composite-cycle interface search

Use exhaustive finite-state analysis as an architecture-search objective for coupled recurrent motifs. Instead of independently tuning a normalization gate and a WTA gate, enumerate their possible interfaces and select couplings that create group action only on a small joint image set, yielding controlled reversible subdynamics embedded in an otherwise dissipative system.

Useful6/10
Difficulty6/10
Novelty9/10
Paper: "More Is Different'' in Neural Circuits: Algebraic Emergence of Effective Theories in Canonical Recurrent Motifs of Biological Neuronal Networks arXiv:2608.30231
Unverified 2026

Reciprocal Paired-Density Network

Represent a neural density or feature field by two positive reciprocal branches whose product is the modeled density, analogous to the forward and backward fields in the paper. Add stochastic branching perturbations to the two branches and train their cross-covariance so that the diagonal paired density matches the target while off-diagonal correlations remain finite-range. This creates a structured alternative to an unconstrained single-field uncertainty representation.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Branching stochastic mechanics. I. Clustering and connected correlations within a branching-process representation of the Schrödinger equation arXiv:2608.29807
Unverified 2026

Mass-Conserving Puncta Router

Insert a differentiable reaction-diffusion layer that converts dense token or pixel features into sparse, spatially coherent routing masks. Two competing orientations form complexes through conserved monomer reservoirs, so local assignments can cluster while opposite assignments mutually exclude one another instead of independently activating at the same location. The layer can be used as a soft-to-hard MoE router, attention-mask generator, or object-part grouping module.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Clustering versus sorting: a mass-conserving reaction-diffusion model of planar polarity puncta arXiv:2608.29679
Unverified 2026

Cumulative-Control Surrogate Encoder

Use cumulative control measures as the input to a neural surrogate instead of the full sequence of control values. For a quasi-static free-boundary system satisfying the paper's average-pressure path-independence assumption, two nonnegative control histories with identical integrals up to time t should produce the same state at t, allowing a smaller training input and fewer distinct control trajectories in the dataset.

Useful6/10
Difficulty4/10
Novelty8/10
Paper: Online Gate-Driven Flow Control in Resin Transfer Moulding Using a Neural-Network Surrogate arXiv:2608.29521
Unverified 2026

Relative-Degree-Gated Passive Neural State Space

Construct a neural state-space model with an explicit first-order input-to-output path instead of forcing every output to depend only on deeply propagated hidden states. Penalize or reject learned linearizations whose transfer matrix has relative degree greater than one, then train a storage-function certificate for the remaining passive dynamics. This preserves the paper's relative-degree compatibility condition while allowing high-order internal memory.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Relative-Degree Wall Restricts Passivity-Based Stability Analysis in Inverter-Dominant Grids arXiv:2608.29474
Unverified 2026

Hard-Saturated Neural Feedback

Build actuator or parameter constraints directly into the neural controller using a differentiable hard-saturation map rather than penalizing violations after the fact. This makes the Lyapunov certificate apply to the actual bounded controller and prevents training from exploiting unrealistically large actions.

Useful6/10
Difficulty3/10
Novelty4/10
Paper: Learning neural controllers for nonlinear systems from data arXiv:2608.29303