Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✓✓ Beats tuned baseline 2026

Reciprocal-Lattice Gauge-Covariant Bloch Network

Build a Bloch-conditioned neural model whose periodic-factor representation transforms covariantly when the supplied Bloch wavenumber is shifted by a reciprocal lattice vector. Either canonicalize q to the first Brillouin zone or augment training with mathematically paired examples whose outputs differ by the exact phase gauge. This prevents the network from learning inconsistent predictions for physically identical Bloch modes.

Useful7/10
Difficulty4/10
Novelty8/10
Paper: Full-field and Bloch-periodic-factor discretizations: Accuracy and phantom modes arXiv:2608.14348
✓✓ Beats tuned baseline 2026

Tiny Local Recurrence with Adaptive Computation

Replace a stack of independently parameterized residual or MLP blocks with a small latent grid or vector repeatedly updated by one shared transition rule. Let the number of updates depend on the current latent state, so easy examples terminate early while hard examples receive more computation, potentially improving parameter efficiency and extrapolation.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Emergent Models: Intelligence from Tiny Substrates arXiv:2608.14019
✓✓ Beats tuned baseline 2026

Displacement-Huber distribution pooling

Replace ordinary Wasserstein or arithmetic pooling of distribution-valued features with a barycenter whose individual quantile displacements are Huberized. Small changes between input distributions remain averaged quadratically, while a corrupted token, expert, graph neighborhood, or augmentation cannot move the pooled distribution arbitrarily far. The module is especially cheap for one-dimensional distributions represented by fixed quantile vectors.

Useful7/10
Difficulty4/10
Novelty7/10
Paper: Huber-Wasserstein barycenters for robust distribution-valued data arXiv:2608.13131
✓✓ Beats tuned baseline 2026

FMM-Accelerated Polyharmonic Neural Field Head

Attach a polyharmonic spline decoder to a coordinate MLP or use it as a standalone neural-field output head over a large set of spatial anchors. The decoder represents the output as a low-degree polynomial trend plus a PHS kernel expansion, while FMM evaluates all anchor-to-query interactions in approximately linear or near-linear cost. When coefficients must be fitted or periodically recalibrated, solve the constrained interpolation system with projected conjugate gradients and a sparse…

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Linear-cost Polyharmonic Spline Interpolation of Arbitrary Degree arXiv:2608.11462
✓✓ Beats tuned baseline 2026

Divergence-Free Spherical Kernel Layer

Build a kernel aggregation layer whose output is a tangent vector field on the unit sphere and whose surface divergence is identically zero by construction. For each source point, use a matrix kernel obtained by applying a surface-rotated gradient in the query variable to a scalar zonal kernel; this is a differential-form version of the paper's matrix-valued construction. The layer can replace attention or message passing when the target dynamics are incompressible, such as spherical fluid…

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Divergence-free interpolation of tangential vector fields via matrix-valued kernels arXiv:2608.05547
✓✓ Beats tuned baseline 2026

Two-Channel Fractal Renormalization Network

Replace ordinary depth-wise feature propagation by a ternary hierarchical block that recursively aggregates three child representations while maintaining separate neutral and defect channels. The block is initialized from the Sierpinski six-vertex recursion, then optionally learns a bounded correction. The neutral channel preserves the paper's cubic mixing law, while the defect channel provides a controlled route for long-range and nonlocal interactions.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Fractal deconfinement and confinement in Sierpinski ice arXiv:2608.02741
✓✓ Beats tuned baseline 2026

Spiderweb Hierarchical Attention

Replace dense token-to-token attention by a multiscale spiderweb communication pattern. Tokens first aggregate upward through a dyadic hierarchy, communicate horizontally only with a small number of cells at the appropriate height, and then receive information broadcast downward. Hyperbolic distance supplies a principled rule for choosing the height at which two tokens interact: nearby tokens interact at fine scales, while far-apart tokens interact through coarse representatives.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Poincaré inequalities on hyperbolic-type spaces arXiv:2608.02369
✓✓ Beats tuned baseline 2026

Holonomy-Twisted Message Passing

Replace an ordinary graph-neural-network edge message by a message transported through a unitary representation of the edge's fundamental-group label. The layer can distinguish globally different holonomy sectors even when the underlying bundles or ordinary graph topology are identical, while inverse edge labels enforce a Hermitian and unitary consistency constraint.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Beyond $K$-Theory: Geometry and Holomorphy in Hyperbolic Band Theory arXiv:2608.01596
✓✓ Beats tuned baseline 2026

Parallel Quadratic Tree Layer

Replace sequential upward message passing in a tree-structured neural module with rake–compress contraction of quadratic latent-state messages. Each node stores a quadratic value function and each edge stores a linear transition or coupling triple; leaf elimination and unary-node compression are implemented as batched Schur complements, followed by a reverse pass that reconstructs node latents and edge outputs. The layer is exact for Gaussian or quadratic latent models and remains…

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Rake-Compress Riccati Recursions for Parallel Scenario-Tree Model Predictive Control arXiv:2608.01332
✓✓ Beats tuned baseline 2026

Symmetry-Preserving Flow Layer

Construct hidden dynamics from permutation-equivariant vector fields and impose antisymmetry through an explicit antisymmetrizing readout. This prevents optimization from learning multiple equivalent copies of the same configuration and makes forbidden symmetry violations exactly zero, rather than merely penalizing them. The design applies to set models, particle systems, graph networks, and architectures handling unordered tokens.

Useful7/10
Difficulty5/10
Novelty4/10
Paper: Spindrift: Learning quantum degeneracy from thermal purity in restricted path integral Monte Carlo arXiv:2607.29590
✓✓ Beats tuned baseline 2026

Interlevel Betti Token Transformer

Represent each input graph by a short ordered sequence of topological tokens rather than a persistence diagram. For each overlapping interval on a shared filtration grid, compute the number of connected components and independent one-dimensional cycles, concatenate these counts with inexpensive graph summary statistics, and process the resulting sequence with a small Transformer. The representation should retain multiscale structural information while avoiding pairwise persistence-diagram…

Useful7/10
Difficulty6/10
Novelty6/10
Paper: TopoFormer: Topology Meets Attention for Graph Learning arXiv:2607.28259
✓✓ Beats tuned baseline 2026

KPZ latent evolution block

Replace an unconstrained recurrent or neural-operator latent transition with a differentiable KPZ cell acting on a spatial latent field. The cell explicitly separates smoothing, nonequilibrium nonlinear steepening, and stochastic forcing, making it suitable for driven dissipative systems and long-horizon roughening that generic networks may fail to reproduce.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Three-Dimensional Kardar--Parisi--Zhang Scaling in Polariton Condensates arXiv:2607.28106
✓✓ Beats tuned baseline 2026

Partial Gromov-Wasserstein Cross-Attention

Replace unconstrained softmax cross-attention with a many-to-many transport matrix whose row and column masses have explicit upper bounds. Compute the attention cost from both feature similarity and pairwise relational disagreement, so a token is attended to only when its relationships to other tokens are jointly compatible. The inequality constraints provide a principled dustbin-free mechanism for ignoring distractor tokens.

Useful7/10
Difficulty6/10
Novelty5/10
Paper: Identifying common backbones of interactions underlying food webs via non-deterministic alignments arXiv:2607.27496
✓✓ Beats tuned baseline 2026

Fermionic circuit message passing

Augment every graph-neural-network edge message with an even commuting channel and a low-dimensional odd anticommuting channel. Contracting odd channels around an edge circuit gives a sign determined by the number of odd edges, while local states with odd incident degree are forced to vanish; this supplies a built-in parity and cycle constraint that ordinary GNNs must learn implicitly.

Useful7/10
Difficulty6/10
Novelty8/10
Paper: Mixed partition functions are exactly the graph parameters of exponentially bounded edge-connection rank arXiv:2607.27198
✓✓ Beats tuned baseline 2026

Tensorized concentration mixing

Replace a dense mixing or attention matrix on tokens arranged on a Cartesian grid by a product of learned or fixed one-dimensional concentration operators. The layer applies one axis operator at a time, reducing parameter and compute cost while enforcing that the global operator is a positive contraction with controlled spectral leakage.

Useful7/10
Difficulty5/10
Novelty5/10
Paper: Tensor factorization and explicit spectral bounds for product-box concentration operators arXiv:2607.26361
✓✓ Beats tuned baseline 2026

Symmetry-Block Neural PDE Solver

Build an implicit or unrolled graph neural operator on a symmetric simplicial mesh, and perform every symmetry-compatible linear solve in a fixed representation-theoretic basis rather than the original edge/face basis. The same basis can be reused for Poisson, Maxwell, diffusion, and learned linear combinations of DEC operators, yielding parallel independent blocks and lower peak memory without changing the discretized solution.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Parallelisation of Discrete Exterior Calculus via Representation Theory on Curved and Three-Dimensional Meshes arXiv:2607.25192
✓✓ Beats tuned baseline 2026

Noisy Scrambling-Front Network

Construct a residual sequence or depth network whose nonnegative influence density follows a discretized noisy Fisher-KPP equation: local influence diffuses, grows when small, saturates at a finite carrying capacity, and receives state-dependent noise. Use this density to gate ordinary feature updates rather than relying only on unconstrained residual additions. The mechanism predicts a measurable propagation speed and an instability boundary, allowing the architecture to be falsified…

Useful7/10
Difficulty6/10
Novelty8/10
Paper: Effective Field Theory of Operator Scrambling from Strong-to-Weak Symmetry Breaking arXiv:2607.24925
✓✓ Beats tuned baseline 2026

Spectral Memory-Lift Ensemble

Replace or augment a recurrent or state-space block with a small ensemble of stable linear memory lifts. Each lift stores a low-dimensional state whose repeated matrix powers generate a structured long-range convolution, and a learned gate mixes the experts using both their current predictions and their slowest block-operator mode. The module can represent multiple memory timescales without explicitly storing a long token history.

Useful7/10
Difficulty6/10
Novelty5/10
Paper: Memory operator ensembles indicate proximity to criticality in simulated AMOC transitions arXiv:2607.24310
✓✓ Beats tuned baseline 2026

Gauge-Patched Local Experts

Represent hidden features with several local orthogonal or unitary frames, and let group-valued transition maps align neighboring experts. Instead of forcing every expert to learn a globally coherent coordinate system, train local experts independently and impose a patching constraint that produces a globally consistent feature field. This is especially suitable for graph neural networks on data with heterogeneous local geometries or mixture-of-experts models whose experts specialize by region.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: A PDE approach to the 2D Yang-Mills measure arXiv:2607.22236
✓✓ Beats tuned baseline 2026

Incidence-Matrix Structured Action Head

Build a policy or prediction head that scores atomic graph entities once and converts those scores into logits for legal composite actions through an instance-specific incidence matrix. The neural parameters are independent of the number of actions and their composition, so the same model can process graphs with different action counts and compatibility structures without padding every action space to a global maximum.

Useful7/10
Difficulty4/10
Novelty6/10
Paper: A Graph-Based Control Interface for Traffic Signals on Heterogeneous Road Networks arXiv:2607.21831
✓✓ Beats tuned baseline 2026

Time-Shell Long-Horizon Decoder

Replace dense pairwise interactions between all forecast horizons with nested time-shell summaries. For sorted horizons, the readout at shell j receives a cumulative embedding of all coefficients or queries assigned to later horizons, reproducing the paper's dependence on products such as \(\Pi_j=\prod_{l>j}e^{\alpha_l}=e^{\sum_{l>j}\alpha_l}\). This gives an \(O(Kd)\) multi-horizon interaction instead of an \(O(K^2d)\) temporal attention block and should work best for weak-memory…

Useful7/10
Difficulty5/10
Novelty8/10
Paper: Dynamical correlation functions of extensive charges after global quantum quenches arXiv:2607.19208
✓✓ Beats tuned baseline 2026

Symmetry-Quotiented Local Correlation Encoder

Replace raw molecular orientation vectors with local scalar features invariant under common three-dimensional rotations and the apolar transformation u_i -> -u_i. Feed these channels to a CNN autoencoder, VAE, or contrastive encoder so that configurations on the same physical symmetry orbit have identical inputs or latent codes. This should improve unsupervised phase discovery without supplying order-parameter labels.

Useful7/10
Difficulty4/10
Novelty5/10
Paper: Representation-Dependent Machine Learning of the Isotropic-Nematic Transition in the Lebwohl-Lasher Model arXiv:2607.16481
✓✓ Beats tuned baseline 2026

Moment-Calibrated Verification Stopping

Replace a fixed-depth all-accept verifier cascade with a depth controller calibrated to the latent distribution of per-instance false-accept rates. The controller should stop when the predicted reliability gain from another gate is smaller than its inference cost, avoiding the severe overconfidence caused by treating correlated verdicts as independent evidence.

Useful7/10
Difficulty4/10
Novelty7/10
Paper: Partially Correlated Verifier Cascades in LLM Harnesses: Concave Log-Odds, Polynomial Reliability, and Blind-Spot Ceilings arXiv:2607.13918
✓✓ Beats tuned baseline 2026

Geometric Feedback Compute Scheduler

Treat unresolved inference items as active threats and allocate a fixed budget of C module evaluations per round. Each evaluation has an item-dependent probability of completing the item, while the scheduler observes only completion or failure after the round. Use fair allocation when completion probabilities are unknown or nearly homogeneous, then switch to a marginal-success greedy policy as feedback estimates become reliable.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Meeting Uncertain Threats with Feedback arXiv:2607.13648