Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

△ Mechanism confirmed, baseline not beaten 2026

Irregular-Domain Fourier Convolution

Use the truncated Fourier representation of an irregular domain as a reusable spectral mask inside an FFT convolution layer. This gives a cheap alternative to point-cloud neighborhood aggregation while explicitly suppressing contributions from outside the physical domain and improving behavior near corners, cusps, and holes.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: "Truncated Fourier Filtering" method for fast and high-order evaluation of integrals and convolutions in general domains arXiv:2608.25264
△ Mechanism confirmed, baseline not beaten 2026

LNC-Compressed Dense Message Passing

For a fixed structured graph, preprocess its adjacency matrix into the linear-time matrix-vector multiplication data structure guaranteed for classes of linear neighborhood complexity. Replace every dense aggregation Y=MX in a GNN by batched queries to this exact data structure, reducing a dense O(n^2d) aggregation to O(nd) after O(n^2) one-time preprocessing. This is especially useful for dense graphs from bounded-clique-width, bounded-expansion, minor-closed, twin-width, or related structured…

Useful7/10
Difficulty7/10
Novelty7/10
Paper: Time-Optimal APSP and Matrix Multiplication in Classes of Linear Neighborhood Complexity arXiv:2608.25212
△ Mechanism confirmed, baseline not beaten 2026

Symplectic Recurrent Block

Use a symplectic Hamiltonian update as a recurrent or state-space neural block, preserving a learned modified energy across many layers or time steps. This targets residual and recurrent architectures where ordinary Euler updates accumulate drift during long rollouts.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Backward error analysis for matrix discretizations of 2-D Euler equations arXiv:2607.09549
△ Mechanism confirmed, baseline not beaten 2026

Cross-Ratio Reversible Lattice Layer

Represent a hidden state as complex-valued points on a two-dimensional lattice and replace unconstrained local updates by the exact harmonic-quadrilateral completion rule from discrete conformal geometry. Given three corners of a plaquette, compute the fourth corner by a Mobius-rational formula enforcing cross-ratio minus one, then use a learned readout or forcing term for task-specific predictions. The layer supplies a hard geometric inductive bias and a directly measurable local constraint…

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Integrability of Cauchy problems for discrete conformal maps and circle patterns arXiv:2607.08901
△ Mechanism confirmed, baseline not beaten 2026

Cross-Channel Scattering Front End

Add a differentiable SNST layer before an EEG classifier or sequence model. For every local channel pair and wavelet band, compute the magnitude of the complex cross-channel analytic response, then average it over a controllable temporal window and concatenate it with ordinary channelwise features. This gives the model an explicit, phase-robust amplitude-coupling representation that is especially useful when labeled training data are scarce.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: Spatial Neighboring Scattering Transform: A Cross-Channel Amplitude Coupling Measure for EEG Connectivity arXiv:2607.08855
△ Mechanism confirmed, baseline not beaten 2026

Trace-Canonical Modular Blocks

Represent a modular neural block as a word over primitive operators and identify operators that commute because they act on disjoint tensor supports. Canonicalize every equivalent word into one representative, allowing architecture search, caching, and parameter sharing to operate on equivalence classes rather than redundant layer orderings.

Useful6/10
Difficulty4/10
Novelty6/10
Paper: Partially-Commutative Polynomial Optimization arXiv:2607.08841
△ Mechanism confirmed, baseline not beaten 2026

Connectivity-Preserving Wedge Token Pooling

Replace a large graph-token set by a smaller set of connected wedge regions generated through adaptive two-seed shortest-path partitions. Each pooled token is the mean of the node features in its region, while the binary partition tree and region sizes are retained for unpooling or skip connections. This provides a deterministic, graph-aware alternative to arbitrary token merging that can be inserted before graph-transformer message passing.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Tonnetz-Driven Graph Wedgelet for Harmonic Complexity Reduction in Music Scores arXiv:2607.08806
△ Mechanism confirmed, baseline not beaten 2026

Spectral Hamiltonian Neuron

Replace a scalar neuron activation with a matrix function of a learned Hamiltonian. Fixed Hermitian interaction operators are combined as a trainable linear Hamiltonian, the activation is applied to its eigenvalues, and the resulting observable is measured on an input quantum state. Noncommuting interaction terms provide a controlled source of expressivity beyond an ordinary scalar neuron.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Canonical quantization of neurons arXiv:2607.05000
△ Mechanism confirmed, baseline not beaten 2026

Bilinear Input-Conditioned Koopman Cell

Replace an unconstrained input-conditioned recurrent transition with a bilinear latent update, so controls modulate a fixed linear latent dynamics matrix through low-rank state-input interactions. The resulting cell preserves the computational simplicity of linear propagation while representing multiplicative effects of actions that an additive control term cannot capture efficiently.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Koopman operator theory: fundamentals, control, and applications arXiv:2607.01819
△ Mechanism confirmed, baseline not beaten 2026

Bounded Commuting Cochain Layer

Replace independently predicted node, edge, and face features on a simplicial mesh by a coupled projection layer that is idempotent, bounded in a mass-matrix norm, and approximately commutes with the discrete exterior derivative. The layer can be inserted after an ordinary graph-neural update and should suppress topologically inconsistent feature components without requiring the downstream network to learn these constraints from data.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: An Approximate Bounded Cochain Projection arXiv:2607.07457
△ Mechanism confirmed, baseline not beaten 2026

Binary-form symmetric-power equivariant layer

Replace an unconstrained feature vector of size n+1 by the coefficients of a homogeneous degree-n binary polynomial and make the layer transform through the irreducible symmetric-power representation of GL_2(R). For n=4 this creates a five-channel equivariant feature block whose transformation law is exact rather than learned through augmentation.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: On 4-dimensional convex projective domains invariant by a lattice of $\mathrm{SL}_2 (\mathbb{R})$ arXiv:2607.07150
△ Mechanism confirmed, baseline not beaten 2026

Fractional Mahalanobis radial head

Replace a binary classifier's unconstrained final logit with a differentiable likelihood-ratio head based on two squared Mahalanobis radii in a learned embedding space. Approximate the shared radial generator with a small fractional-power basis, allowing the head to model heavy-tailed class geometry that an affine QDA logit cannot represent while remaining much smaller than a generic nonlinear head.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Closed-form fractional radial links for elliptical Mahalanobis discriminant analysis arXiv:2607.06089
△ Mechanism confirmed, baseline not beaten 2026

Monotone Singular-Value ICNN Envelope

Replace a generic neural constitutive law or energy model with an ICNN that consumes the positive singular values of a deformation-like matrix and is convex and coordinatewise nondecreasing in those inputs. Train it as a lower approximation to a nonconvex target energy, so the network acts as a computationally cheap sufficient polyconvex-envelope surrogate rather than merely interpolating unstable samples.

Useful6/10
Difficulty4/10
Novelty4/10
Paper: Compression of Polyconvex Envelopes of Isotropic Functions via Monotonic Input Convex Neural Networks arXiv:2607.01055
△ Mechanism confirmed, baseline not beaten 2026

Independent-Simplex Hypergraph Router

Use the paper's edge-to-area incidence structure to choose a small set of geometrically independent simplices instead of processing every possible hyperedge. A greedy rank-increasing router retains a triangle only when its Jacobian adds a new direction, reducing higher-order message-passing cost while preserving diverse geometric information.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: On volume vectors determined by hypergraphs in thin subsets of Euclidean space arXiv:2607.00153
△ Mechanism confirmed, baseline not beaten 2026

Collision-Aware Subset Attention

Replace ordinary one-token-to-one-expert or one-token-to-one-attention routing with a local latent subset router: a pooled observation can be explained by a compatible subset of tokens. Pairwise compatibility scores assign probability to subsets, and each token receives the marginal probability that it belongs to the selected subset. This should help when tokens represent overlapping objects, occluded entities, or multiple features that should be processed jointly.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Gaussian Belief Propagation for Tracking With Unresolved Measurements arXiv:2606.31716
△ Mechanism confirmed, baseline not beaten 2026

Completely Monotone Multiscale Attention Decay

Parameterize a relative-position or lag-decay function as a finite positive mixture of exponentials instead of learning arbitrary attention bias values. The resulting kernel is completely monotone on positive distances, so it is nonnegative, decreasing, and has alternating derivative signs; the mixture provides several learned memory scales without allowing oscillatory or unstable long-range biases.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: Bernstein Functions at Work: Coalescents, Copulas, and Subordination arXiv:2607.04467
△ Mechanism confirmed, baseline not beaten 2026

Markov-coded synchronization-robust latent streams

Replace an independently generated discrete latent sequence or redundancy stream with a symmetric two-state Markov source whose transition probability is tuned or learned. Train the downstream transformer to reconstruct the semantic target after random insertions, while using the paper's insertion-capacity expression to select the latent rate and redundancy budget. The representation should preserve information under timing drift, repeated tokens, and inserted distractors better than iid token…

Useful6/10
Difficulty5/10
Novelty7/10
Paper: The Insertion List-Decoding Capacity and an Improved Bound on the Deletion List-Decoding Capacity arXiv:2607.03989
△ Mechanism confirmed, baseline not beaten 2026

Biclique-free hierarchical attention

Replace dense token-to-token attention by a learned binary relation generated from a small number of hierarchical predicates, while rejecting masks that contain a fixed K_{t,t} biclique. The paper's incidence bound predicts near-linear active edges for these structured relations, giving sparse attention with a measurable worst-case complexity target rather than relying only on average sparsity.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Almost-linear Zarankiewicz bounds in $1$-semi-equational theories arXiv:2608.25464
△ Mechanism confirmed, baseline not beaten 2026

Harmonic Global Latent Channels

Augment a mesh or graph neural network with an explicit low-dimensional channel for topological circulation or flux modes. The network predicts a local gauge-fixed field u and global coefficients a, then reconstructs the physical field as y = u + Ha, so local message passing does not need to synthesize global modes through many layers.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Optimal Control in Hilbert Complex Spaces with Finite Element Exterior Calculus arXiv:2608.25266
△ Mechanism confirmed, baseline not beaten 2026

Positive Garside-factor tokenizer

Replace raw braid-generator sequences by sequences of positive simple Garside factors obtained from the left-greedy normal form. Because powers of \(\Delta\) lie in the Hilden subgroup, they can be removed while preserving the relevant double-coset presentation, reducing non-uniqueness and often shortening the sequence. Feed the resulting factor tokens to a Transformer or sequence classifier, and train it to be invariant to inserted removable \(\Delta\)-powers.

Useful6/10
Difficulty5/10
Novelty9/10
Paper: Every Plat Presentation Admits a Positive Bounded Braid Representative arXiv:2608.24003
△ Mechanism confirmed, baseline not beaten 2026

Finite-Splitting Directional Attention

Construct a directional attention head whose admissible slopes are leaves of an M-adic interval tree with a prescribed finite splitting number. Instead of evaluating all K directions independently at every spatial location, route each query through only the branch decisions of the tree and share feature projections among directions that remain in the same multiscale angular interval.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Directional maximal operators in the plane arXiv:2608.23871
△ Mechanism confirmed, baseline not beaten 2026

PSD Spectral CNN Block

Parameterize a multi-channel two-dimensional convolutional operator through a learned filter bank B, then use the composed operator B*B as the layer response. Its Fourier response is positive semidefinite exactly at every spatial frequency, enabling stable smoothing or diffusion-like residual updates without frequency-grid penalty terms.

Useful6/10
Difficulty4/10
Novelty5/10
Paper: Addendum to "Factoring non-negative operator valued trigonometric polynomials in two variables" arXiv:2608.23073