Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✗ Mechanism failed 2026

Heisenberg latent upsampler

Represent each latent state as a Heisenberg-group element and replace Euclidean interpolation in an upsampling or recurrent transition block by a four-point horizontal refinement plus the exact central signed-area correction. The module preserves the geometry of noncommutative composition, allowing the central latent coordinate to encode path-dependent information that ordinary coordinate-wise interpolation discards.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: A Heisenberg Subdivision Scheme with Central Smoothness Loss arXiv:2607.05446
✓✓ Beats tuned baseline 2026

Gain-Rigid Sparse Attention

Construct a sparse attention or message-passing graph as a gain graph, where every directed edge carries a relative frame label and reverse edges carry the inverse label. Grow the graph using the paper's 2-extension operation: replace two old edges by a new vertex connected to their four endpoints, while preserving the relative gain products. The resulting mask is intended to preserve global information flow under controlled sparsity and to avoid isolated components and brittle bridges commonly…

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Rigidity on compact surfaces through hyperbolic symmetries arXiv:2607.05023
✗ Mechanism failed 2026

Unassembled Adaptive Cell Neural Network

Represent every mesh interface degree of freedom by one feature copy per incident cell, and apply local neural blocks directly to these cell tensors. Enforce inter-cell consistency with valence-weighted averaging only after selected layers or hierarchy transitions, avoiding repeated construction of a global sparse graph or assembled feature vector. This is suited to adaptive quadtrees, octrees, and finite-element neural operators.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Coalesced Matrix-Free Geometric Multigrid on Persistent Cell-Wise Storage arXiv:2607.03413
✓✓ Beats tuned baseline 2026

Agnostic Geometry-Prior Mixer

Train an unconstrained branch and a geometry-aware branch in parallel, then learn how much to trust the analytic branch. This preserves the benefit of explicit geometry on correctly specified tasks while allowing the model to ignore a misleading or irrelevant prior.

Useful6/10
Difficulty3/10
Novelty7/10
Paper: Geometry-Aware R-Structured Kolmogorov-Arnold Networks arXiv:2607.01449
△ Mechanism confirmed, baseline not beaten 2026

Monotone Singular-Value ICNN Envelope

Replace a generic neural constitutive law or energy model with an ICNN that consumes the positive singular values of a deformation-like matrix and is convex and coordinatewise nondecreasing in those inputs. Train it as a lower approximation to a nonconvex target energy, so the network acts as a computationally cheap sufficient polyconvex-envelope surrogate rather than merely interpolating unstable samples.

Useful6/10
Difficulty4/10
Novelty4/10
Paper: Compression of Polyconvex Envelopes of Isotropic Functions via Monotonic Input Convex Neural Networks arXiv:2607.01055
✗ Mechanism failed 2026

Intrinsic-Capacity Feature Bottleneck

Regularize an intermediate neural representation according to its estimated low-dimensional separability capacity instead of its ambient feature width. Learn feature gates or subspace assignments, estimate the union of active supports, and penalize representations whose Cover capacity exceeds a task-dependent target.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Function-Counting Theory for Low-Dimensional Data Structures arXiv:2607.01010
✗ Failed on benchmark 2026

Orthogonal-Rank Contextual Memory

Replace a discrete or one-hot recurrent state table with a low-dimensional vector memory whose event embeddings are orthogonal whenever the corresponding events are mutually exclusive in an input exclusivity graph. The module uses continuous state vectors and can therefore target dimension \(d=\xi(G)\), whereas a discrete state encoding is lower-bounded by \(N\geq\chi(G)\). This should be tested on graph-defined formal-language recognition tasks, where the graph is known and the claimed…

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Quantum Memory Advantage from Contextuality arXiv:2607.00507
✓✓ Beats tuned baseline 2026

Tropical support-restricted PINN

Represent the PINN solution in a restricted polynomial or Taylor basis whose exponent set is supplied by tropical support analysis, instead of asking an MLP to discover the local series structure from scratch. The restriction removes coefficients that cannot occur in the formal solution, reducing trainable degrees of freedom and preventing spurious low-order or singular terms.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Tropical Geometry as a Restricted Architecture for Physics-Informed Neural Networks: Applications in Nonlinear Fluid-Structure Examples arXiv:2607.00237
△ Mechanism confirmed, baseline not beaten 2026

Independent-Simplex Hypergraph Router

Use the paper's edge-to-area incidence structure to choose a small set of geometrically independent simplices instead of processing every possible hyperedge. A greedy rank-increasing router retains a triangle only when its Jacobian adds a new direction, reducing higher-order message-passing cost while preserving diverse geometric information.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: On volume vectors determined by hypergraphs in thin subsets of Euclidean space arXiv:2607.00153
✓✓ Beats tuned baseline 2026

Jacobian-Ranked Simplex Features

Add a differentiable hypergraph layer that converts invariant edge-length features into triangle areas or higher-dimensional simplex volumes before message passing. Select or weight simplices according to the singular values of the length-to-volume Jacobian, so the network receives geometrically independent features rather than many redundant or nearly degenerate measurements.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: On volume vectors determined by hypergraphs in thin subsets of Euclidean space arXiv:2607.00153
✗ Failed on benchmark 2026

GQL Safe Residual Layer

Insert a scalar flux-correction-style limiter after a neural operator predicts a conservative state or residual. Interpolate between a known-admissible baseline state and the learned high-order candidate, choosing the largest coefficient that satisfies a geometric family of linear inequalities encoding positive density, positive pressure, and subluminal velocity. This retains as much of the neural prediction as possible instead of independently clipping physical variables.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: GQL-Based Physical-Constraint-Preserving High-Order Finite Difference Schemes for Special Relativistic Hydrodynamics in Arbitrary Dimensions arXiv:2606.31992
✓✓ Beats tuned baseline 2026

Minkowski-Additive Convex Latents

Store a convex object as a direction-indexed vertex tuple and implement composition of objects through componentwise Minkowski addition and nonnegative scaling. This creates a structured residual or compositional layer where convexification is nonexpansive, making perturbation amplification controllable and avoiding repeated generic geometric optimization.

Useful6/10
Difficulty4/10
Novelty8/10
Paper: Galerkin approximations to the space of convex bodies by polytopes in nondegenerate V-representation arXiv:2608.26615
△ Mechanism confirmed, baseline not beaten 2026

Harmonic Global Latent Channels

Augment a mesh or graph neural network with an explicit low-dimensional channel for topological circulation or flux modes. The network predicts a local gauge-fixed field u and global coefficients a, then reconstructs the physical field as y = u + Ha, so local message passing does not need to synthesize global modes through many layers.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Optimal Control in Hilbert Complex Spaces with Finite Element Exterior Calculus arXiv:2608.25266
✗ Mechanism failed 2026

Sparse Multiscale Kernel-Frame Operator

Replace dense grid tokens or global spectral features with coefficients of compactly supported kernels centered on a nested hierarchy of spatial points. Encode an input field into coarse-to-fine coefficients, apply a neural map to those coefficients, and decode the predicted coefficients at arbitrary query locations; the contribution from each level provides an explicit multiscale output decomposition.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: The Frame Kernel Method for Multiscale Operator Learning arXiv:2608.25084
△ Mechanism confirmed, baseline not beaten 2026

Finite-Splitting Directional Attention

Construct a directional attention head whose admissible slopes are leaves of an M-adic interval tree with a prescribed finite splitting number. Instead of evaluating all K directions independently at every spatial location, route each query through only the branch decisions of the tree and share feature projections among directions that remain in the same multiscale angular interval.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Directional maximal operators in the plane arXiv:2608.23871
✓✓ Beats tuned baseline 2026

Exact Elasticity-Complex Message Passing

Construct a mesh neural network with node, edge, face, and cell feature spaces modeled on the four spaces of the discrete elasticity complex. Replace unconstrained cross-order message passing by fixed incidence and geometric operators whose compositions vanish exactly, so gradient-like, incompatibility-like, and divergence-like features cannot contain algebraically spurious components.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: A Vector-Valued Co-Chain/Chain Complex Associated to the Elasticity Complex arXiv:2608.23829
Unverified 2026

M-convex interaction pruning

Prune higher-order tensor or polynomial interactions while preserving an exchange-closed support instead of independently retaining the largest weights. The resulting sparse interaction pattern avoids arbitrary holes and retains structured substitutions between coordinates, potentially improving parameter efficiency and robustness after pruning.

Useful5/10
Difficulty5/10
Novelty9/10
Paper: Canonical-row Chern flow on Bott--Samelson towers: realizable-volume models for Schubert, Grothendieck, and Lascoux polynomials arXiv:2609.02850
Unverified 2026

Spherical covering initialization

Replace independently random unit-normalized prototypes in a spherical classifier, vector-quantizer, or prototype contrastive head with prototypes selected to cover the sphere evenly. The method directly targets the paper's finding that weak-signal spherical K-means preserves initialization-induced Voronoi structure, reducing redundant prototypes and making early assignments less dependent on random seed.

Useful5/10
Difficulty4/10
Novelty5/10
Paper: A Geometric Analysis of Initialization Bias in Spherical $K$-means in the Weak Signal Regime arXiv:2609.02205
Unverified 2026

Christoffel-Balanced Sparse Attention

Replace a dense or ad hoc sparse attention pattern with a circulant mask generated by the paper's carry word c(m,k). Every query attends to exactly m of k relative positions, and the selected positions are prefix-balanced, avoiding the large gaps and collisions produced by random sparsification. Use several Farey-related slopes across heads or layers to combine local, medium-range, and long-range coverage.

Useful5/10
Difficulty5/10
Novelty7/10
Paper: Farey Structure in Modulo Krinkle Tilings: Mediant Splicing and Generation of Prototiles from a Single Edge arXiv:2609.01270
Unverified 2026

Principal-bundle gauge-fixed Hamiltonian network

Represent a time-dependent Hamiltonian system on the reduced state $(q,t,p_q)$ rather than on the redundant extended state $(q,t,p_q,p_t)$. A neural Hamiltonian section predicts one canonical representative of each affine cotangent fiber, while an optional symmetry loss enforces consistency under transformations that translate time.

Useful5/10
Difficulty5/10
Novelty8/10
Paper: Reduction of symmetric time-dependent Hamiltonian systems I: presymplectic principal $\mathbb{R}$-bundles arXiv:2608.28278
Unverified 2026

Weak-Bounded Riesz Attention

Replace one local spatial aggregation in a CNN or vision transformer with a discretized Riesz potential whose kernel is proportional to $\|x-y\|^{-(n-s)}$. Normalize the layer using the paper's sharp weak-type constant and penalize empirical violations of the resulting tail bound, encouraging nonlocal context without allowing a small set of pixels or tokens to generate arbitrarily large responses.

Useful5/10
Difficulty5/10
Novelty7/10
Paper: Sharp constants for weak estimates of Riesz Potentials when $0<s<\min\{n,2\}$ arXiv:2608.31043
Unverified 2026

Indefinite Grassmann Adapter

Replace an unconstrained low-rank adapter or similarity projection with a learned subspace carrying a prescribed signed metric. The module learns an orthonormal basis U for a k=p+q dimensional subspace, forces the compressed form U^*I_{m,n}U to have p positive and q negative eigenvalues, and uses the resulting pseudo-inner product for signed attention or retrieval scores.

Useful5/10
Difficulty5/10
Novelty8/10
Paper: The Grassmannian of indefinite subspaces arXiv:2608.30249
Unverified 2026

Heisenberg Bracket Positional Encoding

Replace or augment standard sinusoidal or RoPE position features with bracket-quadratic phases $e(-\theta n\lfloor\beta n\rfloor)$ generated by a Heisenberg nilmanifold orbit. Multiple irrational coefficients and output frequencies produce a cheap deterministic encoding whose empirical cross-position correlations should exhibit cancellation instead of the periodic aliasing of rational or finite-frequency encodings.

Useful5/10
Difficulty3/10
Novelty7/10
Paper: Sets whose differences avoid a bracket quadratic arXiv:2608.30078
Unverified 2026

Singularity-Aware Groupoid Transport Layer

Replace a globally shared latent transformation group by a source-dependent collection of valid transformation paths. A feature at latent point z is transported only along paths whose transformed coordinate never reaches the singular locus, while homotopic paths are identified and composable paths are concatenated. This should let an equivariant model represent branched or incomplete symmetries that ordinary group-equivariant layers must discard.

Useful5/10
Difficulty6/10
Novelty8/10
Paper: Lie groupoid integration of singular isometries of the Poincaré disk arXiv:2608.30077