Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

Unverified 2026

Max-times spectral erosion layer

Replace an unconstrained Fourier-domain linear mixer with a bank of positive spectral kernels and a max-times erosion aggregator. For a nonnegative Fourier magnitude f, each kernel produces a quotient response f/psi_k and the layer takes the pointwise supremum over kernels, giving exact positive homogeneity and monotonicity. This is most suitable as a drop-in spectral mixing block in a CNN, vision transformer, or state-space model.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Morphological Representation Theory in the Fourier Inf-Semilattice: Universal Decomposition of Frequency-Domain Deep Learning Operators arXiv:2608.20399
Unverified 2026

Sobolev-optimal frozen analytic features

Replace a trainable shallow hidden layer by a deterministic feature dictionary generated from Chebyshev-spaced scalar parameters and quasi-uniform ridge directions. Train only the output linear map, or use the frozen layer as the first stage of a larger network, thereby eliminating hidden-layer backpropagation while retaining a constructive smooth-function approximation guarantee.

Useful6/10
Difficulty4/10
Novelty6/10
Paper: Sharp Sobolev Approximation on General Domains by Linearized Shallow Networks with Analytic Activations arXiv:2608.18520
Unverified 2026

Parseval Scattering Stem with Certified Depth

Replace the first several convolutional blocks of a small image model with a finite-depth convolution-modulus scattering stem built from a Parseval filter bank. Enforce exact energy accounting and use the paper's polynomial residual law to choose the smallest depth that captures the desired fraction of input energy, avoiding unstable or redundant deep scattering paths.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Universal admissibility for scattering transforms arXiv:2608.18064
Unverified 2026

Dirac-Free Weak Mixed PINN

Build a neural PDE solver that predicts a regularized mixed flux rather than directly fitting a PDE residual containing a Dirac delta. Subtract the explicit radial field generated by the source and train the network with weak constitutive and conservation residuals, so the singularity is represented analytically instead of approximated by a narrow Gaussian.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Mixed Finite Element Methods for a Dirac Source: Divergence-Form Splitting and L^p Error Analysis arXiv:2608.17575
Unverified 2026

Dimension-calibrated contractive branch attractor

Construct a recurrent or generative network from finitely many contractive branches whose hidden-state attractor has a prescribed similarity dimension. The branch contraction ratios determine the target complexity through the equation sum_i r_i^s = 1, while a separation penalty approximates the open set condition and prevents branch collapse or excessive overlap.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: The Moran--Hutchinson formula in semimetric spaces arXiv:2608.16817
Unverified 2026

Negative Bethe-Hessian positional features

Augment node features with eigenvectors corresponding to negative eigenvalues of the Bethe-Hessian H(t,G), rather than using only Laplacian or adjacency positional encodings. The diagonal D-I correction is designed for sparse, locally tree-like graphs and should suppress degree-fluctuation artifacts near the connectivity threshold. Feed the resulting coordinates to a GNN through a learned gate so the model can ignore them when they are uninformative.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: The Bethe-Hessian down to the Percolation Threshold arXiv:2608.16672
Unverified 2026

Scalar Residual Thermodynamic Head

Attach a small temperature-pressure residual head to a pretrained structural encoder instead of relearning the full free-energy surface. Predict one scalar Gibbs free energy and obtain entropy, volume, and other thermodynamic responses by automatic differentiation, enforcing that all outputs derive from a common potential.

Useful6/10
Difficulty4/10
Novelty6/10
Paper: Universal Thermodynamic Interatomic Potentials for Crystalline Materials arXiv:2608.14502
Unverified 2026

Adaptive spectral-gap toroidal encoding

Replace ordinary absolute positional embeddings with coordinates on a learned flat torus and use dual-lattice Fourier characters as positional features. Control the covariance of the coordinate fundamental domain so that the paper's inequality guarantees a lower bound on the smallest nonzero positional frequency, preventing the learned periodic coordinate system from developing arbitrarily weak or nearly constant modes.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Spectral and Isoperimetric Bounds on Flat Tori arXiv:2608.13052
Unverified 2026

Resonance-Gated Triadic Fourier Layer

Replace unconstrained spectral mixing with a three-component triadic interaction whose strength is determined by the quadratic phase mismatch R(xi,xi_1). Near-resonant products receive high weight because their phases remain coherent, while strongly nonresonant products are attenuated. The resonance bandwidth can be fixed from the frequency grid or learned as a positive parameter.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: On solitary wave solutions with two-frequency parameters to the three-component system of quadratic nonlinear Schrödinger equations arXiv:2608.12983
Unverified 2026

Free-boundary neural halting

Replace a fixed confidence-threshold early-exit rule with a finite-horizon optimal-stopping policy over the model's evolving posterior confidence. The controller stops when the calibrated expected terminal error is no greater than the cost plus expected value of executing another neural block, permitting time-dependent and nonmonotone stopping regions.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: When should one stop the most exciting game? Sequential Inference for win-martingales arXiv:2608.12291
Unverified 2026

Bloch-Husimi attention

Replace unconstrained attention score vectors by normalized SU(2) coherent-state responses of a positive operator on an (N+1)-dimensional spin space. Each query produces a smooth bounded response over a fixed spherical grid, while values are aggregated normally. The coherent-state kernel imposes geometric structure and exposes a controllable concentration parameter N.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Isospectral majorization and isoperimetric inequalities for coherent states on the Bloch sphere arXiv:2608.12248
Unverified 2026

Ambiguity-Flat Weyl Feature Layer

Replace a random cyclic filter bank or patch projection with the Weyl–Heisenberg orbit of one normalized learnable prototype. Regularize the prototype so that all nonzero shift and modulation correlations have a large and nearly equal magnitude, maximizing the smallest eigenvalue of the induced feature Gram matrix and preventing poorly observed feature directions.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Uniformly Stable Minimal Weyl--Heisenberg Measurements Approaching the SIC Benchmark arXiv:2608.11850
Unverified 2026

Spherical-design directional heads

Use a fixed spherical t-design as the direction codebook for a directional attention or feature-aggregation module instead of independently sampled random directions. Equal weights provide exact zero mean and isotropic second moments, while exactness for spherical polynomials up to degree t reduces directional aliasing and seed-dependent anisotropy.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Minkowski Polytopes of Spherical Designs: High-Order Isotropy and Quantitative Sphericity arXiv:2608.11570
Unverified 2026

Deep-spectrum community features

Replace the usual top-eigenvector positional encoding in a graph neural network with a density-selected spectral subspace. The selector explicitly searches below the leading eigenvectors, where community information may survive after latent geometric modes have consumed the largest eigenvalues. The selected coordinates can be concatenated to node features or used as a bias in graph attention.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Spectral graph clustering with inhomogeneous latent geometry arXiv:2608.11321
Unverified 2026

Equivariant Critical-Mode Branch Seeding

When a symmetry-frequency block becomes critical, initialize or perturb the network specifically along its critical representation rather than injecting isotropic noise into all hidden channels. This creates trainable branches for the symmetry patterns predicted by the bifurcation calculation and can expose useful periodic solutions that ordinary symmetry-preserving training fails to reach.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Local and Global Equivariant Bifurcation for Periodic Weyl and Riesz Fractional Equations arXiv:2608.11101
Unverified 2026

Measure-Lifted Entropy Amplifier

Replace a deterministic latent state with a probability measure over latent states, represented by particles or weighted prototypes. Apply the learned latent transition to every particle, so one base trajectory map induces a dynamics on distributions; use an entropy-preservation or entropy-growth regularizer to prevent collapse of the ensemble. The mechanism predicts that any positive base-state trajectory entropy can generate unbounded distinguishability in the ideal measure space through…

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Entropies of compact subsets and supported measures arXiv:2608.09702
Unverified 2026

Geodesic Low-Rank Latent Bottleneck

Replace a Euclidean low-rank latent decoder with a geodesic factor decoder on a Riemannian manifold. A learned location α provides the component center, a small set of tangent loading vectors V captures anisotropic variation, and latent coefficients z generate curved manifold-valued features through the exponential map. Multiple such decoders can form a mixture-of-geodesic-experts layer for multimodal representations.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Mixture of Geodesic Factor Analyzers on Riemannian Homogeneous Spaces arXiv:2608.06971
Unverified 2026

KPZ Directed-Polymer Attention

Replace independent Gaussian attention noise or unconstrained token routing with a directed-polymer path distribution over positions and layers. The router aggregates exponentially many monotone paths through temporally correlated random edge scores, producing heavy-tailed but spatially coherent routing and preventing attention from collapsing onto a single token. The paper's t^{2/3} wandering and t^{1/3} free-energy fluctuations become measurable diagnostics and tunable targets rather than…

Useful6/10
Difficulty5/10
Novelty7/10
Paper: KPZ Superdiffusion of Local Correlators in Diffusive Random Quantum Circuits arXiv:2608.06459
Unverified 2026

Intersection Euler Interaction Token

Compute a compact multiscale interaction signature between colored point clouds and append it to a point-cloud or multimodal neural network as a learned interaction token. The signature captures separated, overlapping, and higher-order enclosing configurations while remaining invariant to rigid transformations.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: The Intersection Euler Characteristic Profile: Euler Calculus and Stability for Topological Interaction of Ball Unions arXiv:2608.06180
Unverified 2026

Power-Law Volterra Memory

Add a causal memory branch whose weights are generated by the paper's power-type Volterra kernel rather than learned independently at every lag. Learn or softly constrain the exponents so the model can select rough short-memory behavior or smoother long-memory behavior while using only a few parameters. The branch can be implemented as a truncated causal convolution, a multiresolution approximation, or a recurrent state-space realization.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Small ball probabilities and Chung's law of the iterated logarithm for Gaussian Volterra processes with power-type kernels arXiv:2608.05679
Unverified 2026

Automaton-Gated Objective Switching

Augment a neural policy with deterministic DFA states for the task objective and safety constraint, then select among objective-specific policy heads using those states. Before either target is reached, execute a mixed policy; after one target is reached, switch permanently to the policy specialized for the remaining target.

Useful6/10
Difficulty4/10
Novelty5/10
Paper: Optimal Constrained sc-LTL Planning in MDPs via Switching Policies arXiv:2608.05021
Unverified 2026

Prototype Distance-Field Safety Layer

Store a finite library of successful robot configurations or action-conditioned waypoints and construct a smooth soft minimum of their distances. Use the negative distance gradient as a structured action prior, add a learned residual policy, and pass the combined action through a quadratic-program safety layer. This gives a neural controller an explicit attraction basin toward demonstrated solutions while preventing violations of known state constraints.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Grasp Execution Without a Planner: Configuration-Space Grasp Distance Fields with Certified Safety & Guaranteed Quality arXiv:2608.00600
Unverified 2026

Contracting Countable-Branch Router

Construct a routed neural state update from a collection of branch maps whose inverse-style refinement operators are uniformly contractive. The discrete routing sequence acts as an expandable symbolic code, while contraction makes the continuous state associated with a long routing history insensitive to initialization and earlier perturbations. Use a finite active top-\(K\) set during training, but retain an expandable branch table so the model can represent increasingly complex or rare modes.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Infinite-Piecewise Expanding Maps: Chaos, Ergodicity and Invariant-Set Complexity arXiv:2608.00398
Unverified 2026

Signed-base fractal positional features

Augment standard Transformer positional embeddings with coordinates generated by the paper's signed-base digit expansion. Previous binary digits determine the sign and scale of later contributions, while a two-state Markov chain controls correlations between digits. This supplies multiscale positional structure using a small number of transition and base parameters.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: Fractal random variables defined by probability distributions of digits of their $G_2$-representation having two bases with different signs arXiv:2607.29327