Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

Unverified 2026

Learned fractional-scale convolution

Replace a single Laplacian or fixed diffusion regularizer in a CNN with a finite positive mixture of fractional Laplacians at several orders. The resulting module separately controls short-range smoothing and long-range spatial coupling, while positivity preserves a dissipative energy and avoids the unstable behavior of arbitrary signed mixtures.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Ground state solutions for Hartree type equations driven by superposition operators and Pohozaev Identity arXiv:2607.19076
Unverified 2026

Hard-Rod Symmetry Invariant Module

Construct a scalar feature or critic for oscillator-based neural dynamics that is invariant under the transformations imposed by free harmonic motion and elastic collisions. For finite-size rods, the module should represent only quantities compatible with common oscillator-phase rotations and momentum permutations, preventing a learned world model from inventing coordinate-dependent pseudo-conserved quantities that disappear after collisions.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Absence of hidden analytic conserved quantities in harmonically confined rods arXiv:2607.18872
Unverified 2026

Anisotropic Anharmonic Diffusion Layer

Insert a learnable semigroup layer that evolves features according to a positive operator combining frequency damping and spatially varying confinement. Unlike isotropic Gaussian smoothing, the layer can damp selected frequencies differently along different axes and can suppress activations in learned spatial regions, while the positive-semigroup construction prevents amplification.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Sharp Time-Decay Estimates for Fractional Heat Semigroups Associated with Polynomial Anharmonic Oscillators arXiv:2607.17580
Unverified 2026

Parity-Constrained Signed Propagation

Replace the unsigned adjacency used by a deep message-passing network with a signing selected from an affine family that makes designated short even cycles unbalanced. Search this family for a small even-power trace, which acts as a proxy for a smaller spectral radius and suppresses explosive long-range propagation. The signing can be fixed before training, so the method adds no per-example inference cost.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Parity families and a kernel-averaged L-function for near-Ramanujan signings arXiv:2607.17343
Unverified 2026

Gaussian Simplex Classification Head

Replace the unconstrained final classifier with equal-norm regular-simplex class directions and train it under explicit isotropic Gaussian feature noise. At fixed signal energy and equal class priors, the paper's Gaussian-max theorem predicts that this geometry maximizes finite-noise maximum-likelihood decoding probability, making it a concrete candidate for robust classification heads.

Useful6/10
Difficulty4/10
Novelty4/10
Paper: Stochastic Domination of Gaussian Maxima: A Resolution of the Weak Simplex Conjecture arXiv:2607.14087
Unverified 2026

Hermite-Schatten spectral layer

Replace a dense learned linear operator on continuous or image features by a truncated Hermite projection expansion whose coefficients are directly regularized in a Schatten-p norm. The layer becomes a structured low-rank operator, while the radial Hermite-Laguerre correspondence provides an analytically tractable parameterization and an exact spectral penalty.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Quantitative Fourier Restriction Estimates for Weyl Operators: Fourier-Support Dependence and Lower Bounds arXiv:2607.13697
Unverified 2026

Decoration-Iteration Graph Coarsening

Construct a graph-neural layer that analytically eliminates fast auxiliary nodes inside repeated decorated motifs and replaces each motif by an effective edge or hyperedge. The effective interaction is computed from the log-partition function of the eliminated variables, while a residual neural correction can model violations of the assumed local motif structure.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Thermal phase transitions in a mixed-spin Ising model on the Lieb lattice: Exact results beyond zero magnetic field arXiv:2607.11661
Unverified 2026

Boundary-Only Cell-Complex Network

Parameterize a cell-complex neural network by features on p-cells and derive lower-dimensional boundary features using the cellular boundary map over F2. For a 2D square complex, neighboring plaquette bits determine each link feature through XOR, reproducing the paper's exact gauge-law reconstruction and preventing the network from representing inconsistent open boundary configurations.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: $p$-Form Gauge Dynamics and Digital Quantum Simulation -- Flux and Cosmological Constant Neutralization arXiv:2607.10950
Unverified 2026

Branch-Gated Ambiguity Layer

Replace a single smooth inverse predictor near detected ambiguity boundaries with multiple prediction branches and a soft gate. The gate is trained to preserve distinct decompositions rather than forcing the network to interpolate through a thin high-curvature transition layer, while a Jacobian or curvature penalty identifies unresolved ambiguity regions.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Discontinuous Prior-Mode Sections and the Geometry of Ambiguity in Intrinsic Image Decomposition arXiv:2607.10321
✗ Mechanism failed 2026

Algebraic Pinch-Curve Spectral Layer

Replace a dense learnable Fourier multiplier with a low-parameter multiplier concentrated near the common zero set of two polynomial constraint symbols. A linear constraint together with a cubic constraint can produce straight or curved frequency loci, allowing the network to represent directional long-range structure while using far fewer spectral parameters than a full 3D frequency grid.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Symmetry-Protected Pinch Curves in Classical Spin Liquids arXiv:2607.09470
✗ Mechanism failed 2026

Resolution-adaptive spectral front end

Replace a fixed Fourier or spectral resolution in a neural operator or sequence model with a data-adaptive spectral cutoff. Keep only modes whose estimated signal energy exceeds the noise-amplification and discretization floor implied by the available number of trajectories and samples per trajectory. This should reduce overfitting to high-frequency sensor noise and preserve accuracy when the same model is deployed at different sampling resolutions.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: The Cost of Discretization in Functional Linear Regression: Minimax Rates and Adaptation arXiv:2607.09350
△ Mechanism confirmed, baseline not beaten 2026

Bounded Commuting Cochain Layer

Replace independently predicted node, edge, and face features on a simplicial mesh by a coupled projection layer that is idempotent, bounded in a mass-matrix norm, and approximately commutes with the discrete exterior derivative. The layer can be inserted after an ordinary graph-neural update and should suppress topologically inconsistent feature components without requiring the downstream network to learn these constraints from data.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: An Approximate Bounded Cochain Projection arXiv:2607.07457
✗ Mechanism failed 2026

Support-Budgeted Hamming Polynomial Layer

Replace the first dense layer on q-ary categorical features by a Fourier interaction layer containing only monomials whose coordinate support is at most s. Use a Bohnenblust–Hille-inspired quasi-norm on coefficients, separately for each interaction order, to prevent a small number of high-order interactions from dominating the output. The resulting model has an explicit interaction-order knob and can be tested against a dense MLP at matched parameter count.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Support-Sensitive Bohnenblust-Hille Inequalities and Local Invariants on Hamming Schemes arXiv:2607.05594
✗ Mechanism failed 2026

Bernstein resolvent activation

Replace an unconstrained scalar activation or nonnegative gate with a finite positive mixture of rational Bernstein basis functions. The learned function is monotone and concave on the nonnegative half-line, while its derivatives have controlled alternating signs; this can prevent pathological feature amplification and gives an interpretable shape prior. Use the paper's sharp exponent restriction τ≤1/2 rather than treating the power as an arbitrary hyperparameter.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: Riccati Reductions for Modified Bessel Ratios: Bernstein Positivity, Exact Certificates, and Transfer Obstructions arXiv:2607.05538
✗ Mechanism failed 2026

Fourier-Calibrated Nonlocal Feature Gradient

Augment a CNN with a nonlocal feature-gradient branch that compares each feature vector with a kernel-weighted neighborhood rather than using only pointwise or local convolutional interactions. Regularize this branch using the paper's Fourier multiplier energy, which penalizes feature oscillations according to the kernel spectrum and approaches an ordinary local-gradient operator as the interaction radius tends to zero.

Useful6/10
Difficulty4/10
Novelty6/10
Paper: A Variational Nonlocal Phase-Field Model for Dynamic Fracture in Elastic Solids arXiv:2607.01881
✗ Mechanism failed 2026

Totally-positive bidiagonal mixer

Replace a dense channel or token-mixing matrix with a product of positive bidiagonal factors, so information propagates through a controlled sequence of local couplings rather than arbitrary signed interactions. Initialize the factors from the paper's barycentric-subdivision factorization, then learn positive diagonal and off-diagonal parameters; the resulting map is structured, parameter-efficient, and constrained to remain totally positive.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Total positivity of transformation matrices for uniform subdivisions arXiv:2607.01577
✓✓ Beats tuned baseline 2026

Agnostic Geometry-Prior Mixer

Train an unconstrained branch and a geometry-aware branch in parallel, then learn how much to trust the analytic branch. This preserves the benefit of explicit geometry on correctly specified tasks while allowing the model to ignore a misleading or irrelevant prior.

Useful6/10
Difficulty3/10
Novelty7/10
Paper: Geometry-Aware R-Structured Kolmogorov-Arnold Networks arXiv:2607.01449
△ Mechanism confirmed, baseline not beaten 2026

Monotone Singular-Value ICNN Envelope

Replace a generic neural constitutive law or energy model with an ICNN that consumes the positive singular values of a deformation-like matrix and is convex and coordinatewise nondecreasing in those inputs. Train it as a lower approximation to a nonconvex target energy, so the network acts as a computationally cheap sufficient polyconvex-envelope surrogate rather than merely interpolating unstable samples.

Useful6/10
Difficulty4/10
Novelty4/10
Paper: Compression of Polyconvex Envelopes of Isotropic Functions via Monotonic Input Convex Neural Networks arXiv:2607.01055
✗ Mechanism failed 2026

Intrinsic-Capacity Feature Bottleneck

Regularize an intermediate neural representation according to its estimated low-dimensional separability capacity instead of its ambient feature width. Learn feature gates or subspace assignments, estimate the union of active supports, and penalize representations whose Cover capacity exceeds a task-dependent target.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Function-Counting Theory for Low-Dimensional Data Structures arXiv:2607.01010
✓✓ Beats tuned baseline 2026

Minkowski-Additive Convex Latents

Store a convex object as a direction-indexed vertex tuple and implement composition of objects through componentwise Minkowski addition and nonnegative scaling. This creates a structured residual or compositional layer where convexification is nonexpansive, making perturbation amplification controllable and avoiding repeated generic geometric optimization.

Useful6/10
Difficulty4/10
Novelty8/10
Paper: Galerkin approximations to the space of convex bodies by polytopes in nondegenerate V-representation arXiv:2608.26615
✗ Mechanism failed 2026

Wick-Matching Polynomial Interaction Layer

Replace an unconstrained high-order polynomial interaction module with features generated by Gaussian matrix contractions and their exact Wick expansion. The resulting interactions are sums of products of power-sum invariants, with coefficients fixed by perfect-matching counts, providing a low-parameter inductive bias for permutation- or orthogonal-structured data.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Stable Symmetric Series, Differential Operators, and Jack Deformations arXiv:2608.25651
Unverified 2026

Certified Radical Coordinate Layer

Replace Fourier or sinusoidal one-dimensional coordinate features with a trainable radical layer f(x)=sum_i c_i sqrt(P_i(x)), where every P_i is a strictly positive quadratic. A nonzero scalar output formed by such a layer has at most 2n distinct real zeros, providing an explicit bound on sign changes and suppressing uncontrolled ringing. Use the radical features as an input embedding for a conventional MLP or neural implicit field.

Useful5/10
Difficulty4/10
Novelty8/10
Paper: Estimating the number of real zeros of linear combinations of radicals of polynomials arXiv:2609.02871
Unverified 2026

Principal-bundle gauge-fixed Hamiltonian network

Represent a time-dependent Hamiltonian system on the reduced state $(q,t,p_q)$ rather than on the redundant extended state $(q,t,p_q,p_t)$. A neural Hamiltonian section predicts one canonical representative of each affine cotangent fiber, while an optional symmetry loss enforces consistency under transformations that translate time.

Useful5/10
Difficulty5/10
Novelty8/10
Paper: Reduction of symmetric time-dependent Hamiltonian systems I: presymplectic principal $\mathbb{R}$-bundles arXiv:2608.28278
Unverified 2026

Level-Set Balanced Sparse Mixer

Partition activations into dyadic magnitude bands and allocate sparse connectivity separately to heavy and diffuse coordinates. Protect high-magnitude coordinates with more reliable connections while using randomized flat connectivity for the many small coordinates, keeping the total number of nonzeros fixed.

Useful5/10
Difficulty6/10
Novelty7/10
Paper: Level-set entropy and sparse randomized embeddings arXiv:2607.23017