Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

△ Mechanism confirmed, baseline not beaten 2026

MCIS Safety Shield for Neural Controllers

Compute an inner approximation of the states from which a neural controller can keep the plant inside a prescribed safe domain indefinitely, then use the resulting regulation map as a safety shield around the network. At each state, the network proposes an action, but the shield projects or replaces it with an action certified to remain in the invariant set.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: Computing the Maximal Controlled Invariant Set for Neural Network Control Systems arXiv:2608.07908
✗ Failed on benchmark 2026

Generator-Flow Equivariance Training

Use discovered infinitesimal generators to create small continuous transformations of hidden states and force a neural predictor to commute with those transformations. This converts symmetry discovery into self-supervised augmentation without prespecifying a group, canonical coordinates, or hand-designed equivariant layers.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: LieStoNet: Learning Lie Symmetries from Spatiotemporal Data for Stochastic Dynamical Systems arXiv:2608.01582
△ Mechanism confirmed, baseline not beaten 2026

Bundle-Glued Neural Field

Represent a field on a manifold with one neural network per chart, while enforcing the exact transition law between chart outputs on overlaps. This avoids the artificial requirement that one coordinate frame work globally and should improve learning on spherical, periodic, or otherwise topologically nontrivial domains. Use an augmented Lagrangian rather than only a pointwise penalty so chart compatibility is enforced strongly without requiring identical local parameterizations.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Finite element discretization of Yang--Mills connections arXiv:2608.02108
✗ Mechanism failed 2026

Dyadic Resolvent Filter Layer

Replace a single polynomial graph filter or dense inverse with a multiscale layer that applies unitary propagations at dyadic times and combines them according to the dyadic resolvent identity. For a symmetric graph operator, this implements a stable rational spectral filter that can selectively retain or suppress frequency bands while exposing logarithmic multiscale structure.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Dyadic Resolvent Representations of Self-Adjoint Operators: Propagator Expansions, Spectral Measures, and Zeta Functions arXiv:2607.21278
△ Mechanism confirmed, baseline not beaten 2026

Invariant-domain learned reconstruction

Insert a neural local reconstruction into a finite-volume or graph-based simulator, but hard-cap its contribution so every reconstructed state remains in the physical admissible set. The network learns accuracy-sensitive gradients or stencil weights; a deterministic limiter, rather than a penalty loss, guarantees positive density and pressure for arbitrary network outputs.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Guarantees by Construction for Learned Finite Volume Schemes on Steady Supersonic Flow arXiv:2607.20171
△ Mechanism confirmed, baseline not beaten 2026

Monotone CDT autoencoder bottleneck

Build an autoencoder whose decoder outputs a monotone quantile function rather than an unconstrained spatial field. The latent representation can be compressed with POD or a neural bottleneck in CDT space, while the decoder guarantees valid transport maps and therefore avoids negative densities, mass drift, and spurious oscillations common in unconstrained reduced-order neural decoders.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Reduced Order Modeling of One-Dimensional Conservative PDEs via the Cumulative Distribution Transform arXiv:2607.17066
△ Mechanism confirmed, baseline not beaten 2026

Noise-Triggered Latent Rank Adaptation

Use the recursive errors-in-variables subspace spectrum as a controller for the width of a latent SSM rather than fixing the state dimension in advance. Neurons or state channels are added when corrected covariance eigenvalues rise above the noise floor and pruned when they remain below it, producing a model-order-adaptive recurrent architecture for nonstationary streams.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: A recursive subspace based method for errors-in-variables model identification of time-varying systems arXiv:2607.17065
△ Mechanism confirmed, baseline not beaten 2026

Constraint-preserving DAE neural block

Build a neural dynamical block whose hidden state contains differential variables and Lagrange multipliers, with a singular descriptor matrix enforcing constraints during propagation. This avoids the drift and ill-conditioning that can arise when exact constraints are represented only by a penalty term.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Contour integral methods and structured perturbations for linear differential-algebraic equations arXiv:2607.12628
Unverified 2026

Quadratic-budget Toeplitz long-range layer

Replace a dense translation-invariant interaction matrix with a positive-definite Toeplitz kernel K_n(e^f) whose log-spectrum is parameterized by a small number of Fourier coefficients with 1/|k| decay. Use the paper's explicit quadratic term as a spectral-volume budget, allowing long-range structure while discouraging uncontrolled determinant growth and ill-conditioning. Subtracting this term from a log-determinant regularizer leaves a residual intended to capture higher-order deviations from…

Useful6/10
Difficulty6/10
Novelty7/10
Paper: On Toeplitz determinants with slow Fourier decay arXiv:2608.13182
Unverified 2026

Sobolev-Orthogonal MLP Features

Replace raw polynomial or Fourier-like features in a small MLP with basis functions orthonormal under a Sobolev inner product that jointly measures feature magnitude and input derivative magnitude. This explicitly controls feature smoothness while preserving decorrelation, potentially improving conditioning and reducing the need for large derivative-regularization coefficients.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: A Riemann-Hilbert representation for Sobolev orthogonal polynomials arXiv:2608.08397
Unverified 2026

Anisotropic Anharmonic Diffusion Layer

Insert a learnable semigroup layer that evolves features according to a positive operator combining frequency damping and spatially varying confinement. Unlike isotropic Gaussian smoothing, the layer can damp selected frequencies differently along different axes and can suppress activations in learned spatial regions, while the positive-semigroup construction prevents amplification.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Sharp Time-Decay Estimates for Fractional Heat Semigroups Associated with Polynomial Anharmonic Oscillators arXiv:2607.17580
✗ Mechanism failed 2026

Fourier-Calibrated Nonlocal Feature Gradient

Augment a CNN with a nonlocal feature-gradient branch that compares each feature vector with a kernel-weighted neighborhood rather than using only pointwise or local convolutional interactions. Regularize this branch using the paper's Fourier multiplier energy, which penalizes feature oscillations according to the kernel spectrum and approaches an ordinary local-gradient operator as the interaction radius tends to zero.

Useful6/10
Difficulty4/10
Novelty6/10
Paper: A Variational Nonlocal Phase-Field Model for Dynamic Fracture in Elastic Solids arXiv:2607.01881
Unverified 2026

Adjoint spectral opening for stable pooling

Construct a learnable spectral pooling block as an erosion followed by its adjoint dilation, making the resulting opening idempotent, increasing, and anti-extensive. The block can suppress frequencies outside a learned passband while guaranteeing that applying it twice does not continue changing the representation, which is useful in multi-stage CNN pyramids and U-Net skip paths.

Useful5/10
Difficulty6/10
Novelty6/10
Paper: Morphological Representation Theory in the Fourier Inf-Semilattice: Universal Decomposition of Frequency-Domain Deep Learning Operators arXiv:2608.20399
Unverified 2026

Convex-order stochastic expert layer

Replace a deterministic mixture-of-experts residual block with K population-indexed stochastic expert states coupled through a graphon matrix. The layer uses a shared drift and expert-dependent diffusion, while an empirical convex-order penalty makes later representations more dispersed than a reference representation without permitting a mean shift.

Useful5/10
Difficulty5/10
Novelty7/10
Paper: Convex order preservation for graphon mean-field systems arXiv:2608.19576
Unverified 2026

Mean-Curvature Relaxation Layer

Insert a small number of differentiable graphical mean-curvature-flow steps between a neural network's raw vector-field prediction and its task loss. The relaxation performs geometry-aware smoothing rather than isotropic Gaussian smoothing, and it can enforce fixed boundary values after every step.

Useful5/10
Difficulty5/10
Novelty7/10
Paper: Well-posedness for the mean curvature flow on the half-space and on bounded domains arXiv:2608.08901
Unverified 2026

Complex-stretched resonance layer

Insert a fixed or learnable complex coordinate stretch outside the region where a neural operator models the physical interaction, so outgoing waves are damped and resonant states become ordinary discrete eigenmodes on a finite grid. Train the network with eigenvalue or resolvent losses computed after the stretch, while preserving the physical field in the interior region.

Useful5/10
Difficulty6/10
Novelty7/10
Paper: Dirac resonances as non-self-adjoint eigenvalues arXiv:2607.26166
Unverified 2026

LU-Preconditioned Orthogonal Weight Retraction

Periodically project a rectangular neural-network weight matrix onto an approximately orthonormal-column matrix using LU-preconditioned CholeskyQR rather than ordinary QR or a polar iteration. Pivoted LU handles badly scaled and nearly dependent columns, while Householder orthogonalization of the LU factor produces a triangular preconditioner that makes the subsequent Cholesky step safer in fp16 or bfloat16.

Useful5/10
Difficulty6/10
Novelty5/10
Paper: RCLUPPr: a new randomized CholeskyQR with LU preconditioning arXiv:2607.15561