Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

Unverified 2026

Cross-Ratio Reversible Lattice Layer

Represent a hidden state as complex-valued points on a two-dimensional lattice and replace unconstrained local updates by the exact harmonic-quadrilateral completion rule from discrete conformal geometry. Given three corners of a plaquette, compute the fourth corner by a Mobius-rational formula enforcing cross-ratio minus one, then use a learned readout or forcing term for task-specific predictions. The layer supplies a hard geometric inductive bias and a directly measurable local constraint…

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Integrability of Cauchy problems for discrete conformal maps and circle patterns arXiv:2607.08901
Unverified 2026

Cross-Channel Scattering Front End

Add a differentiable SNST layer before an EEG classifier or sequence model. For every local channel pair and wavelet band, compute the magnitude of the complex cross-channel analytic response, then average it over a controllable temporal window and concatenate it with ordinary channelwise features. This gives the model an explicit, phase-robust amplitude-coupling representation that is especially useful when labeled training data are scarce.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: Spatial Neighboring Scattering Transform: A Cross-Channel Amplitude Coupling Measure for EEG Connectivity arXiv:2607.08855
Unverified 2026

Trace-Canonical Modular Blocks

Represent a modular neural block as a word over primitive operators and identify operators that commute because they act on disjoint tensor supports. Canonicalize every equivalent word into one representative, allowing architecture search, caching, and parameter sharing to operate on equivalence classes rather than redundant layer orderings.

Useful6/10
Difficulty4/10
Novelty6/10
Paper: Partially-Commutative Polynomial Optimization arXiv:2607.08841
Unverified 2026

Connectivity-Preserving Wedge Token Pooling

Replace a large graph-token set by a smaller set of connected wedge regions generated through adaptive two-seed shortest-path partitions. Each pooled token is the mean of the node features in its region, while the binary partition tree and region sizes are retained for unpooling or skip connections. This provides a deterministic, graph-aware alternative to arbitrary token merging that can be inserted before graph-transformer message passing.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Tonnetz-Driven Graph Wedgelet for Harmonic Complexity Reduction in Music Scores arXiv:2607.08806
Unverified 2026

Task-Visible Axis Pruning

Prune hidden units only after testing whether their nonlinear gate is task-visible and downstream-used. For ReLU, a unit is removable when its preactivation does not cross zero on the task patch or its outgoing weight column is zero; this is a more structural criterion than weight magnitude and can be applied during width search or post-training compression.

Useful6/10
Difficulty3/10
Novelty5/10
Paper: Contravariance Theory: Strong Alignment for Minimal Solutions to Hard Tasks arXiv:2607.08561
Unverified 2026

Wide-tree invariant alignment layer

Replace a purely pairwise embedding similarity used for set alignment with a sum of rooted-tree contraction scores. Each tree feature aggregates products of several coordinate-level interactions and can preserve correspondence information under an unknown orthogonal transformation, allowing matching from moderate correlation rather than nearly identical embeddings. Use the resulting score matrix for Hungarian matching, contrastive loss, or a differentiable Sinkhorn assignment.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: High-Dimensional Procrustes Matching via Tree Counts arXiv:2607.08538
Unverified 2026

Finite-group relative message passing

Use the quotient group's generator classes as a finite relation vocabulary and tie message functions by group displacement instead of by individual graph edges. This creates a compact, exactly consistent relation-aware GNN that can recognize repeated local structure and transfer parameters across graph instances sharing the same Cayley geometry.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Minimal Isometric Embeddings of Graphs into Cayley Graphs of Finite Abelian Groups arXiv:2607.07920
Unverified 2026

Neighborhood-separator attention

Construct a sparse token-interaction graph from attention affinities and recursively split it by removing the closed neighborhoods of a small set of separator tokens. Separator tokens retain global communication, while the resulting anti-adjacent components perform local attention independently, reducing quadratic attention and KV-cache costs. The induced Erdos-Pósa theorem supplies a structural diagnostic: graphs with few anti-adjacent long-cycle or theta packings should admit small…

Useful6/10
Difficulty7/10
Novelty8/10
Paper: Induced Erdős--Pósa property for long holes, long thetas, and beyond arXiv:2607.07697
Unverified 2026

Innovation-Compensated Latent Policy

In a partially observed reinforcement-learning or model-based control agent, expose the state-estimator innovation to the action head through a dedicated residual feedback branch. The policy produces a nominal action from the estimated latent state, while a learned innovation-compensation branch corrects actions when observations disagree with predicted latent dynamics. This explicitly separates nominal policy behavior from estimation-induced corrections and should help during fast transients…

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Revisiting Certainty Equivalence: The Structural Coupling Between Estimation and Control in Underactuated Nonlinear Systems arXiv:2607.07276
Unverified 2026

Age-conditioned semi-Markov router

Augment a neural router with the age of its current expert or latent regime and use an age-dependent hazard to determine when switching is likely. Unlike ordinary token-wise softmax routing, the router can learn non-geometric residence times, suppressing unstable expert oscillations while still allowing rapid transitions when the current regime becomes inappropriate.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Semi-Markovian switching in a fluctuating harmonic trap: An age-structured formulation arXiv:2607.05173
Unverified 2026

Spectral Hamiltonian Neuron

Replace a scalar neuron activation with a matrix function of a learned Hamiltonian. Fixed Hermitian interaction operators are combined as a trainable linear Hamiltonian, the activation is applied to its eigenvalues, and the resulting observable is measured on an input quantum state. Noncommuting interaction terms provide a controlled source of expressivity beyond an ordinary scalar neuron.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Canonical quantization of neurons arXiv:2607.05000
Unverified 2026

Bilinear Input-Conditioned Koopman Cell

Replace an unconstrained input-conditioned recurrent transition with a bilinear latent update, so controls modulate a fixed linear latent dynamics matrix through low-rank state-input interactions. The resulting cell preserves the computational simplicity of linear propagation while representing multiplicative effects of actions that an additive control term cannot capture efficiently.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Koopman operator theory: fundamentals, control, and applications arXiv:2607.01819
Unverified 2026

Bounded Commuting Cochain Layer

Replace independently predicted node, edge, and face features on a simplicial mesh by a coupled projection layer that is idempotent, bounded in a mass-matrix norm, and approximately commutes with the discrete exterior derivative. The layer can be inserted after an ordinary graph-neural update and should suppress topologically inconsistent feature components without requiring the downstream network to learn these constraints from data.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: An Approximate Bounded Cochain Projection arXiv:2607.07457
Unverified 2026

Binary-form symmetric-power equivariant layer

Replace an unconstrained feature vector of size n+1 by the coefficients of a homogeneous degree-n binary polynomial and make the layer transform through the irreducible symmetric-power representation of GL_2(R). For n=4 this creates a five-channel equivariant feature block whose transformation law is exact rather than learned through augmentation.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: On 4-dimensional convex projective domains invariant by a lattice of $\mathrm{SL}_2 (\mathbb{R})$ arXiv:2607.07150
Unverified 2026

Hankel Residual Observer

Attach a model-free residual-dynamics observer to a neural multi-step forecaster. Instead of asking the network to relearn persistent periodic or autoregressive disturbances, maintain a Hankel dictionary of recent forecast errors and use ridge reconstruction to predict the next residual sequence online. Add the predicted residual to the network forecast with a confidence-dependent correction gain.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: Model-Free Disturbance Observer with Online Modification: Listening to MFDOOM arXiv:2607.07082
Unverified 2026

Reversible Mealy Token Mixer

Replace a recurrent token-mixing operation with a finite-state carrier scan over binary or quantized token features. The local transition table is constrained to conserve a scalar token weight and to be bijective, making the mixer reversible, constant-memory, and less prone to activation drift than a generic recurrent layer.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Invariant Measures for Soliton Systems Generated by Mealy Automata arXiv:2607.06942
Unverified 2026

Observability-Gated Spectral Phase Initialization

Add a preprocessing or differentiable synchronization layer that estimates one unit-modulus complex phase per graph node or data view from noisy pairwise relative-phase observations. Initialize the phases with a leading-eigenvector method, fix the global phase gauge, and allow nonlinear refinement only when the estimated perturbation is small relative to the observable Jacobian margin. This replaces random initialization for rotation-alignment modules and should reduce bad local minima caused…

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Spectral Initialization and Certification for Power System Angle Estimation arXiv:2607.06762
Unverified 2026

Branch-Length Polynomial Fingerprint

Add a deterministic, branch-length-aware fingerprint to a rooted-tree neural encoder using the paper's symmetric product recursion. The fingerprint distinguishes child multisets structurally and incorporates every edge length, providing information that ordinary sum or mean message passing can lose.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: Polynomial encoding of rooted trees with branch lengths arXiv:2607.06591
Unverified 2026

Diffeomorphic gauge-fixing layer

Insert a differentiable spatial canonicalization module before a neural dynamics model. It estimates a smooth invertible coordinate transformation that places each input field in a common gauge relative to a reference template, predicts the next state in that gauge, and maps predictions back to the original coordinates. The module should reduce the need for the dynamics network to relearn identical laws under many smooth spatial reparameterizations.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: The Right Space for Dynamics: Numerics with Diffeomorphism Equivariance arXiv:2607.06536
Unverified 2026

Compressed threshold-overlap Gram layer

Represent each k-element object by a vector in dimension \(r=\binom{n-2(k-s)}{s}\), and use a PSD Gram matrix to encode the rule that pairs with intersection smaller than s have zero similarity while pairs with intersection at least s have nonzero similarity. Insert this representation into set encoders, graph neural networks, or overlap-aware attention instead of allocating one feature for every s-subset.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Minimum-rank parameters of complements of threshold Kneser graphs arXiv:2607.06480
Unverified 2026

Periodic CMV Unitary Recurrent Layer

Replace a dense recurrent transition matrix with a periodic CMV-style product of alternating local 2x2 unitary cores. The transition is exactly norm-preserving, has O(n) trainable parameters under periodic tying, and can be applied through local factor operations rather than stored as an n-by-n matrix. Use turnover refactorization when changing the ordering or boundary connection of cores, enabling a compact cyclic unitary state-space layer.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Fast computation of eigenvalues of periodic CMV matrices arXiv:2607.06400
Unverified 2026

Fractional Mahalanobis radial head

Replace a binary classifier's unconstrained final logit with a differentiable likelihood-ratio head based on two squared Mahalanobis radii in a learned embedding space. Approximate the shared radial generator with a small fractional-power basis, allowing the head to model heavy-tailed class geometry that an affine QDA logit cannot represent while remaining much smaller than a generic nonlinear head.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Closed-form fractional radial links for elliptical Mahalanobis discriminant analysis arXiv:2607.06089
Unverified 2026

Lipschitz-Free Metric Pooling

Replace coordinate-wise mean pooling of metric-valued items with a finite representation of their free integral. Each item x in a pointed metric space M is represented through evaluations of learned Lipschitz probes, and the pooled feature is the weighted integral of those probe values. A dual Lipschitz critic estimates the free-space norm of differences between pooled groups, making the representation sensitive to metric geometry while remaining permutation-invariant.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Analytic integration of metric-valued functions in Lipschitz free spaces arXiv:2607.06049
Unverified 2026

Adaptive CUR Neural Layer

Replace a dense weight matrix by a cross approximation built from selected rows and columns rather than by a conventional truncated SVD. Periodically refresh the selected indices using residual leverage scores, warm-starting from the previous factorization so that the compressed layer can track weight changes during fine-tuning.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: A Sub-linear Low-Rank Solver for Poisson's Equation using Machine Learning Frameworks for GPU Acceleration arXiv:2607.06021