Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

Unverified 2026

Volume-Weighted Hodge Convolution

Replace the ordinary combinatorial Hodge propagation in a simplicial neural network with a geometry-induced weighted Hodge Laplacian built from Euclidean simplex volumes. The operator preserves the harmonic/topological subspace while changing the positive spectrum according to the shape and scale of the simplices, allowing message passing to distinguish geometrically meaningful cells that have identical incidence patterns.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Geometry-Induced Hodge Stars on Rips and Dowker--Rips Complexes arXiv:2607.18692
Unverified 2026

Concave Heterogeneous Prototype Layer

Replace ordinary k-means-style prototype assignment with a distance-decay capture layer whose scale varies across samples, tokens, or classes. Train with a cooperative concave surrogate over prototype centers and anneal toward hard nearest-prototype assignment; this explicitly preserves useful gradients for multiple nearby prototypes while retaining sparse facility-like behavior at inference.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: When Is Heterogeneous Distance-Decay Facility Location Tractable? A Structural Classification, Exact Methods, and a Real-World Study arXiv:2607.16764
Unverified 2026

RLCT-Certified Singular Attention

Use an anisotropic singular relative-position kernel in attention or graph message passing, with its exponent constrained by the paper's local integrability threshold. The module can represent sharper directional interactions than an RBF while providing an explicit certificate that its spatial gradient belongs to a chosen L^p space.

Useful6/10
Difficulty5/10
Novelty9/10
Paper: Geometric Criteria for Morrey Admissibility via the Real Log-Canonical Threshold arXiv:2607.14991
Unverified 2026

Second-Order Sinh-Gordon Implicit Layer

Insert a differentiable implicit layer that maps boundary features to an interior latent field by solving a discrete sinh-Gordon equation. The paper's second-order convergence result motivates using a symmetric five-point discretization and a damped Newton solve rather than asking a neural network to learn the entire interior field directly.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Approximation of solutions of the sinh-Gordon equation $Δu -\sinh(2u)=0$ by hyperbolic orthogonal ring patterns arXiv:2607.14348
Unverified 2026

Threshold-Projection Recurrent Memory

Replace or augment a recurrent cell with multiple hysteresis memory branches whose states remain unchanged while the input stays within a branch-specific radius, then move toward the current input only when that radius is exceeded. The resulting cell has explicit persistence and bounded state changes, giving it an inductive bias for temporal hysteresis and reducing the need for the network to learn long-term memory behavior from scratch.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: Accounting for Hysteresis and Eddy Currents in Finite Element Simulations of Ferromagnetic Laminated Cores using a Recurrent Neural Network arXiv:2607.14321
Unverified 2026

Spectral latent geometry for sparse attention

Build a sparse graph by thresholding normalized token or item inner products, then use the leading eigenvectors of its centered adjacency matrix as geometric features or a low-rank attention-logit bias. The graph avoids storing all pairwise similarities, while the paper's spectral bound supplies a concrete signal-to-noise test for deciding whether the resulting embedding is trustworthy.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Spectral Concentration and Recovery in Sparse High-Dimensional Random Geometric Graphs arXiv:2607.14304
Unverified 2026

Gaussian Simplex Classification Head

Replace the unconstrained final classifier with equal-norm regular-simplex class directions and train it under explicit isotropic Gaussian feature noise. At fixed signal energy and equal class priors, the paper's Gaussian-max theorem predicts that this geometry maximizes finite-noise maximum-likelihood decoding probability, making it a concrete candidate for robust classification heads.

Useful6/10
Difficulty4/10
Novelty4/10
Paper: Stochastic Domination of Gaussian Maxima: A Resolution of the Weak Simplex Conjecture arXiv:2607.14087
Unverified 2026

Arcosh-convex PINN output

Replace the usual unconstrained PINN output u_theta(x) with a latent field w_theta(x), and reconstruct the physical solution as u_theta(x) = -2 log cosh(w_theta(x)). Train w_theta to be convex while enforcing the Liouville or real two-Hessian PDE residual and the boundary condition u = 0. The transformation automatically gives u less than or equal to zero, matching the target solutions, while convexity supplies a strong global shape prior.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Strict Convexity for Solution of Liouville-Type Dirichlet Problems arXiv:2607.12849
Unverified 2026

Hessian-guided orthogonal sparsification

Train a square orthogonal neural mixer while maximizing its entrywise fourth-power concentration. When optimization reaches a non-permutation stationary configuration, explicitly test rank-two row or column rotations and take a rotation with positive exact second variation, using the paper's constructive saddle-escape mechanism.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Local Maxima of the Entrywise $\ell_4$ Norm on the Orthogonal Group arXiv:2607.12431
Unverified 2026

Horizontal Symmetry Quotient Layer

Replace explicit quotient construction by a differentiable projection that removes learned group-orbit directions from both source and target features. The paper's reduction argument shows that a closed level constraint makes the restricted form horizontal, so the network can operate on invariant coordinates while retaining a measurable residual for symmetry leakage.

Useful6/10
Difficulty6/10
Novelty5/10
Paper: Reduction of relative multisymplectic manifolds arXiv:2607.12350
Unverified 2026

Contour Levy-Area Feature Branch

Add a geometric branch that converts ordered image contours into truncated signatures and feeds first-order displacement and second-order antisymmetric area features into the detector backbone. The area channel captures orientation and enclosed-region structure that ordinary edge magnitude or convolutional filters may miss, making the module suitable for thin cracks, scratches, bent boundaries, and small industrial defects.

Useful6/10
Difficulty4/10
Novelty6/10
Paper: Rough Path Signature-Guided Geometry Augmentation for Few-Shot Industrial Surface Defect Detection arXiv:2607.12245
Unverified 2026

Algebraic-Invariant Residual Layer

Represent a rational-like feature transformation with an auxiliary state y constrained by polynomial equations G(x,y)=0, and update x and y jointly along the tangent space of that constraint manifold. This creates residual blocks in which nonlinear feature identities remain consistent over many layers or time steps, reducing auxiliary-variable drift and potentially stabilizing rational activations and implicit recurrent dynamics.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Algebraic Invariant Quadratization Schemes for Cahn--Hilliard Equations arXiv:2607.11569
Unverified 2026

Submodular Budget Router

Replace independent top-k MoE routing with a submodular polyhedral allocation over experts. A learned set function assigns a marginal gain to each additional expert allocation, so the router exhibits diminishing returns and can enforce global capacity constraints rather than making unrelated per-token choices. The allocation is obtained by sorting marginal gains, giving a fast greedy router with piecewise-linear routing regions.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Extended generalized permutahedra, and cointeracting bialgebras arXiv:2607.10683
Unverified 2026

Exact simplex-lattice quantization

Replace independent coordinate rounding of a fixed-sum vector with nearest-point quantization in the projected integer lattice A_n^*. The quantized vector preserves the zero-sum constraint exactly, while the globally optimal rounding correction accounts for the aggregate residual induced by projection.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: A Faster Closest-Point Algorithm for the $A_n^*$ Lattices arXiv:2607.10479
Unverified 2026

Branch-Gated Ambiguity Layer

Replace a single smooth inverse predictor near detected ambiguity boundaries with multiple prediction branches and a soft gate. The gate is trained to preserve distinct decompositions rather than forcing the network to interpolate through a thin high-curvature transition layer, while a Jacobian or curvature penalty identifies unresolved ambiguity regions.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Discontinuous Prior-Mode Sections and the Geometry of Ambiguity in Intrinsic Image Decomposition arXiv:2607.10321
Unverified 2026

Projected Play-State Memory

Turn a recurrent or state-space memory into a constrained hereditary state: the latent state remains in a learned convex domain, and only input motion that reaches the boundary changes the plastic component. This creates a nonexpansive, rate-independent memory that should suppress unstable state growth and make the representation depend on meaningful cumulative changes rather than arbitrary update frequency.

Useful6/10
Difficulty4/10
Novelty5/10
Paper: Optimal history encoding for elastic-plastic hereditary laws: Sharp input and constitutive approximation arXiv:2607.09974
Unverified 2026

Swarmalator Token Organizer

Augment each token or graph node with a periodic latent position x_i and phase θ_i, then evolve these variables before attention or message passing. Tokens with similar phase attract in x, while tokens with similar position synchronize in θ, producing self-organized groups without an externally specified clustering objective. The coupling strengths J and K provide interpretable controls for aggregation and synchronization, and their sweep should expose the paper's four collective regimes and…

Useful6/10
Difficulty5/10
Novelty8/10
Paper: A solvable normal form for coupled swarmalators arXiv:2607.09810
✗ Failed on benchmark 2026

Inverse-Square Fractional Attention

Replace or augment geometric attention on spatial or point-cloud tokens with a positive fractional kernel containing the paper's inverse-square origin factor. This gives tokens near a designated singular center a controlled increase in receptive-field influence while preserving a scale-invariant distance decay, which may help models represent cusp-like fields, radial singularities, and multiscale spatial interactions.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Sharp and Endpoint Two-Weight Fractional Integral Estimates for Schr"odinger Operators with Inverse-Square Potentials arXiv:2607.09585
✗ Mechanism failed 2026

Algebraic Pinch-Curve Spectral Layer

Replace a dense learnable Fourier multiplier with a low-parameter multiplier concentrated near the common zero set of two polynomial constraint symbols. A linear constraint together with a cubic constraint can produce straight or curved frequency loci, allowing the network to represent directional long-range structure while using far fewer spectral parameters than a full 3D frequency grid.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Symmetry-Protected Pinch Curves in Classical Spin Liquids arXiv:2607.09470
△ Mechanism confirmed, baseline not beaten 2026

Cross-Ratio Reversible Lattice Layer

Represent a hidden state as complex-valued points on a two-dimensional lattice and replace unconstrained local updates by the exact harmonic-quadrilateral completion rule from discrete conformal geometry. Given three corners of a plaquette, compute the fourth corner by a Mobius-rational formula enforcing cross-ratio minus one, then use a learned readout or forcing term for task-specific predictions. The layer supplies a hard geometric inductive bias and a directly measurable local constraint…

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Integrability of Cauchy problems for discrete conformal maps and circle patterns arXiv:2607.08901
✗ Failed on benchmark 2026

Task-Visible Axis Pruning

Prune hidden units only after testing whether their nonlinear gate is task-visible and downstream-used. For ReLU, a unit is removable when its preactivation does not cross zero on the task patch or its outgoing weight column is zero; this is a more structural criterion than weight magnitude and can be applied during width search or post-training compression.

Useful6/10
Difficulty3/10
Novelty5/10
Paper: Contravariance Theory: Strong Alignment for Minimal Solutions to Hard Tasks arXiv:2607.08561
✓✓ Beats tuned baseline 2026

Observability-Gated Spectral Phase Initialization

Add a preprocessing or differentiable synchronization layer that estimates one unit-modulus complex phase per graph node or data view from noisy pairwise relative-phase observations. Initialize the phases with a leading-eigenvector method, fix the global phase gauge, and allow nonlinear refinement only when the estimated perturbation is small relative to the observable Jacobian margin. This replaces random initialization for rotation-alignment modules and should reduce bad local minima caused…

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Spectral Initialization and Certification for Power System Angle Estimation arXiv:2607.06762
✗ Mechanism failed 2026

Diffeomorphic gauge-fixing layer

Insert a differentiable spatial canonicalization module before a neural dynamics model. It estimates a smooth invertible coordinate transformation that places each input field in a common gauge relative to a reference template, predicts the next state in that gauge, and maps predictions back to the original coordinates. The module should reduce the need for the dynamics network to relearn identical laws under many smooth spatial reparameterizations.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: The Right Space for Dynamics: Numerics with Diffeomorphism Equivariance arXiv:2607.06536
△ Mechanism confirmed, baseline not beaten 2026

Fractional Mahalanobis radial head

Replace a binary classifier's unconstrained final logit with a differentiable likelihood-ratio head based on two squared Mahalanobis radii in a learned embedding space. Approximate the shared radial generator with a small fractional-power basis, allowing the head to model heavy-tailed class geometry that an affine QDA logit cannot represent while remaining much smaller than a generic nonlinear head.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Closed-form fractional radial links for elliptical Mahalanobis discriminant analysis arXiv:2607.06089