Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✓✓ Beats tuned baseline 2026

Cyclic Momentum Latents

Add a learnable cyclic-coordinate mechanism to latent dynamics so that selected latent coordinates do not enter the Hamiltonian and their conjugate momenta become conserved. This provides an explicit dimensionality-discovery and invariance bias, encouraging the model to represent nuisance or symmetry directions compactly instead of spending independent dynamics capacity on them.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Generating Physically Plausible Parachute Dynamics with Deep Generative Modeling arXiv:2607.12143
✓✓ Beats tuned baseline 2026

Newton-Polytope Convex Network

Build a positively homogeneous convex network by representing every intermediate unit as a compact polytope and composing units with Minkowski sums, convex-hull unions, and positive dilations. This gives an explicitly convex and monotone architecture whose geometric complexity can be controlled independently of the number of sampled linear pieces, potentially producing smaller ICNNs for structured convex functions.

Useful7/10
Difficulty7/10
Novelty7/10
Paper: Tropical Circuits with Scalar Multiplication Gates arXiv:2607.11540
✓✓ Beats tuned baseline 2026

Multiplicative Manifold Unscented Recurrent Cell

Replace the Euclidean hidden-state update of a recurrent or state-space neural network with a mixed manifold state containing a rotation component and Euclidean features. Propagate uncertainty with sigma points in tangent error coordinates, retract rotational perturbations through the exponential map, and compute the training loss from the predicted covariance. This avoids invalid rotations and captures second-order curvature effects that a first-order EKF-style recurrent cell misses at large…

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Fully Multiplicative Attitude and Orbit Determination for Deep space Navigation arXiv:2607.10072
✓✓ Beats tuned baseline 2026

Cyclic Multirate Neural State Space

Replace interpolation of heterogeneous sensor streams by a phase-indexed recurrent or state-space network with period M, where M is the least common multiple of the sensor sampling periods. The network applies a distinct transition for each phase while using a fixed cyclic phase update, preserving timing structure and allowing missing observations to enter only when their phase-specific sensor is available.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Cyclic Reformulation-Based Identification and Polytopic Uncertainty Modeling for Multirate Systems arXiv:2607.09194
✓✓ Beats tuned baseline 2026

Control-Cost Laguerre MoE Routing

Route tokens to experts using Laguerre cells defined by the minimum control energy needed to move a token embedding to each expert prototype, rather than by Euclidean distance or an unconstrained learned router logit. Per-expert dual weights deform the cells so that minibatch routing follows prescribed expert capacities, giving a geometrically interpretable alternative to auxiliary load-balancing losses.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Control Laguerre Tessellation: Semi-discrete Optimal Transport Over Control Systems arXiv:2607.09139
✓✓ Beats tuned baseline 2026

Smith-normal-form Cayley positional encoding

Replace heuristic graph positional encodings with exact finite-abelian-group coordinates derived from edge-class increments and cycle constraints. Relative positions become group differences, allowing a graph transformer to share parameters across repeated generator displacements while retaining exact path consistency and compact cyclic coordinates.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Minimal Isometric Embeddings of Graphs into Cayley Graphs of Finite Abelian Groups arXiv:2607.07920
✓✓ Beats tuned baseline 2026

Conformal-symplectic sandwich layer

Replace an unconstrained one-step transition network with a symmetric damping–symplectic-core–damping composition. The damping strength is one learned scalar rate and is applied through positive exponential diagonal factors, so every step has a known contraction law while the neural core models nonlinear conservative transport.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: CSympNet-ID: conformal-symplectic map learning for linearly damped Hamiltonian systems arXiv:2607.03339
✓✓ Beats tuned baseline 2026

Woodbury Data-Consistency Layer for Multiplexed Unrolling

Replace the usual gradient-descent or conjugate-gradient data-fidelity step in an unrolled reconstruction network with an exact Woodbury proximal layer for grouped multiplexed measurements. The layer can be inserted between learned denoising blocks and should provide stronger measurement consistency at a fixed number of unrolled stages, while avoiding inner iterative linear solves.

Useful7/10
Difficulty4/10
Novelty6/10
Paper: Plug-and-Play Volumetric Reconstruction for Compressive Sensing Light-Sheet Microscopy arXiv:2607.01654
✓✓ Beats tuned baseline 2026

Differentiable R-Function Geometry Gate

Attach an analytic geometry gate to a KAN or MLP so that known feasible regions, exclusions, and unions are represented by differentiable implicit functions instead of being learned only from samples. Use R-conjunctions for intersections and R-disjunctions for unions, then convert the signed support score into a soft gate that modulates the prediction.

Useful7/10
Difficulty4/10
Novelty6/10
Paper: Geometry-Aware R-Structured Kolmogorov-Arnold Networks arXiv:2607.01449
✓✓ Beats tuned baseline 2026

Spectral-gap local mixing

Replace a dense graph-attention or token-mixing matrix by a resolvent-like interaction operator and truncate it to graph neighborhoods whose radius is selected from an estimated spectral gap. Unlike fixed-window sparse attention, the sparsity level is tied to a measurable stability parameter and has an explicit exponential tail criterion.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Waveguiding in systems of high contrast resonators: Theory and fast computations arXiv:2608.26906
✓✓ Beats tuned baseline 2026

Local Characteristic Residual Gating

Transform local neural residuals into the Ripa model's characteristic coordinates before spatial aggregation, apply a mode-dependent gate based on neighboring characteristic jumps, and transform back. This lets the model damp oscillatory acoustic or equilibrium-mode corrections near discontinuities without globally smoothing every feature.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Fifth-Order Well-Balanced Path-Conservative A-WENO Scheme for the Ripa Model arXiv:2607.09293
✓✓ Beats tuned baseline 2026

Epoch-Frozen Masked Low-Rank Candidate Encoder

Add an online low-rank reconstruction layer before a recommender or ranking MLP. It estimates a latent action-feature subspace from masked candidate vectors, freezes that subspace for an epoch, reconstructs each partially observed candidate in the latent coordinates, and feeds only those coordinates to the predictor. The method is most promising when the ambient candidate dimension d is large but the effective rank m is small and missingness is not too severe.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Stochastic Linear Bandits with Partially Observed Actions arXiv:2607.08971
✓✓ Beats tuned baseline 2026

Finite-group relative message passing

Use the quotient group's generator classes as a finite relation vocabulary and tie message functions by group displacement instead of by individual graph edges. This creates a compact, exactly consistent relation-aware GNN that can recognize repeated local structure and transfer parameters across graph instances sharing the same Cayley geometry.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Minimal Isometric Embeddings of Graphs into Cayley Graphs of Finite Abelian Groups arXiv:2607.07920
✓✓ Beats tuned baseline 2026

Reversible Mealy Token Mixer

Replace a recurrent token-mixing operation with a finite-state carrier scan over binary or quantized token features. The local transition table is constrained to conserve a scalar token weight and to be bijective, making the mixer reversible, constant-memory, and less prone to activation drift than a generic recurrent layer.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Invariant Measures for Soliton Systems Generated by Mealy Automata arXiv:2607.06942
✓✓ Beats tuned baseline 2026

Observability-Gated Spectral Phase Initialization

Add a preprocessing or differentiable synchronization layer that estimates one unit-modulus complex phase per graph node or data view from noisy pairwise relative-phase observations. Initialize the phases with a leading-eigenvector method, fix the global phase gauge, and allow nonlinear refinement only when the estimated perturbation is small relative to the observable Jacobian margin. This replaces random initialization for rotation-alignment modules and should reduce bad local minima caused…

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Spectral Initialization and Certification for Power System Angle Estimation arXiv:2607.06762
✓✓ Beats tuned baseline 2026

Branch-Length Polynomial Fingerprint

Add a deterministic, branch-length-aware fingerprint to a rooted-tree neural encoder using the paper's symmetric product recursion. The fingerprint distinguishes child multisets structurally and incorporates every edge length, providing information that ordinary sum or mean message passing can lose.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: Polynomial encoding of rooted trees with branch lengths arXiv:2607.06591
✓✓ Beats tuned baseline 2026

Gain-Rigid Sparse Attention

Construct a sparse attention or message-passing graph as a gain graph, where every directed edge carries a relative frame label and reverse edges carry the inverse label. Grow the graph using the paper's 2-extension operation: replace two old edges by a new vertex connected to their four endpoints, while preserving the relative gain products. The resulting mask is intended to preserve global information flow under controlled sparsity and to avoid isolated components and brittle bridges commonly…

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Rigidity on compact surfaces through hyperbolic symmetries arXiv:2607.05023
✓✓ Beats tuned baseline 2026

Critical-Tail Multiscale Mixer

Add a fixed or weakly parameterized residual mixer whose interaction between sequence positions at distance \(r\) is proportional to \(1/(r\log^2 r)\). Instead of truncating the kernel at a short radius, represent its heavy tail with dyadic distance bands and compute each band using prefix sums or block pooling, giving every token access to arbitrarily distant context at roughly \(O(L\log L)\) cost.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Long-range interactions and Anderson localisation for one-dimensional high-contrast resonator chain arXiv:2607.04971
✓✓ Beats tuned baseline 2026

Kolmogorov-Lie Unitary Layer

Build an input-conditioned unitary transformation as an ordered product of exponentials of anti-Hermitian matrices, with each factor controlled by a univariate function of one input coordinate or one learned scalar projection. This replaces a dense multivariate matrix-valued controller with separable scalar nonlinearities while preserving exact unitarity at every forward pass.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Quantum Kolmogorov--Arnold representation theorem for continuous unitary-valued maps arXiv:2607.03187
✓✓ Beats tuned baseline 2026

Singularity-Enriched Neural Ansatz

Add an explicit local power-law singular basis to a neural field near mixed Dirichlet-Neumann junctions, allowing the neural network to learn only the smoother remainder. Use the predicted or fitted singular exponent to concentrate collocation points near the junction. This directly targets the regularity bottleneck identified by the paper, where increasing polynomial degree or network capacity cannot overcome a convergence cap under uniform resolution.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: A Unified CutFEM Formulation for Finite-Strain Elasticity: Energy Minimisation and Corner Singularities arXiv:2607.02334
✓✓ Beats tuned baseline 2026

Fractional Boundary-Factored Neural Solver

For a fractional Dirichlet problem, replace a free coordinate network N_theta(x) with u_theta(x)=d(x)^a N_theta(x), where d(x)=dist(x,boundary) and 0<a<1 is the fractional order. Train the regular quotient v_theta=u_theta/d^a=N_theta and use a weighted gradient loss that reflects the paper's boundary estimate.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: The structure of solution spaces for fractional-order operators, with gradient estimates arXiv:2607.02312
✓✓ Beats tuned baseline 2026

Agnostic Geometry-Prior Mixer

Train an unconstrained branch and a geometry-aware branch in parallel, then learn how much to trust the analytic branch. This preserves the benefit of explicit geometry on correctly specified tasks while allowing the model to ignore a misleading or irrelevant prior.

Useful6/10
Difficulty3/10
Novelty7/10
Paper: Geometry-Aware R-Structured Kolmogorov-Arnold Networks arXiv:2607.01449
✓✓ Beats tuned baseline 2026

Tropical support-restricted PINN

Represent the PINN solution in a restricted polynomial or Taylor basis whose exponent set is supplied by tropical support analysis, instead of asking an MLP to discover the local series structure from scratch. The restriction removes coefficients that cannot occur in the formal solution, reducing trainable degrees of freedom and preventing spurious low-order or singular terms.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Tropical Geometry as a Restricted Architecture for Physics-Informed Neural Networks: Applications in Nonlinear Fluid-Structure Examples arXiv:2607.00237
✓✓ Beats tuned baseline 2026

Jacobian-Ranked Simplex Features

Add a differentiable hypergraph layer that converts invariant edge-length features into triangle areas or higher-dimensional simplex volumes before message passing. Select or weight simplices according to the singular values of the length-to-volume Jacobian, so the network receives geometrically independent features rather than many redundant or nearly degenerate measurements.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: On volume vectors determined by hypergraphs in thin subsets of Euclidean space arXiv:2607.00153