Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

Unverified 2026

Projected Play-State Memory

Turn a recurrent or state-space memory into a constrained hereditary state: the latent state remains in a learned convex domain, and only input motion that reaches the boundary changes the plastic component. This creates a nonexpansive, rate-independent memory that should suppress unstable state growth and make the representation depend on meaningful cumulative changes rather than arbitrary update frequency.

Useful6/10
Difficulty4/10
Novelty5/10
Paper: Optimal history encoding for elastic-plastic hereditary laws: Sharp input and constitutive approximation arXiv:2607.09974
Unverified 2026

Swarmalator Token Organizer

Augment each token or graph node with a periodic latent position x_i and phase θ_i, then evolve these variables before attention or message passing. Tokens with similar phase attract in x, while tokens with similar position synchronize in θ, producing self-organized groups without an externally specified clustering objective. The coupling strengths J and K provide interpretable controls for aggregation and synchronization, and their sweep should expose the paper's four collective regimes and…

Useful6/10
Difficulty5/10
Novelty8/10
Paper: A solvable normal form for coupled swarmalators arXiv:2607.09810
Unverified 2026

Collider-Aware DAG Variational Network

Replace independent uncertainty heads in a branching neural network with a structured variational posterior whose non-root node distributions condition on jointly sampled latent states of all parents. This allows collider evidence to explain away upstream uncertainty: evidence at a child can alter the posterior over several parent branches instead of leaving their uncertainties artificially independent. The approach can be implemented as a stochastic DAG network and trained with an evidence…

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Deep Gaussian Processes on Directed Acyclic Graphs arXiv:2607.09645
✗ Failed on benchmark 2026

Inverse-Square Fractional Attention

Replace or augment geometric attention on spatial or point-cloud tokens with a positive fractional kernel containing the paper's inverse-square origin factor. This gives tokens near a designated singular center a controlled increase in receptive-field influence while preserving a scale-invariant distance decay, which may help models represent cusp-like fields, radial singularities, and multiscale spatial interactions.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Sharp and Endpoint Two-Weight Fractional Integral Estimates for Schr"odinger Operators with Inverse-Square Potentials arXiv:2607.09585
△ Mechanism confirmed, baseline not beaten 2026

Symplectic Recurrent Block

Use a symplectic Hamiltonian update as a recurrent or state-space neural block, preserving a learned modified energy across many layers or time steps. This targets residual and recurrent architectures where ordinary Euler updates accumulate drift during long rollouts.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Backward error analysis for matrix discretizations of 2-D Euler equations arXiv:2607.09549
✗ Mechanism failed 2026

Algebraic Pinch-Curve Spectral Layer

Replace a dense learnable Fourier multiplier with a low-parameter multiplier concentrated near the common zero set of two polynomial constraint symbols. A linear constraint together with a cubic constraint can produce straight or curved frequency loci, allowing the network to represent directional long-range structure while using far fewer spectral parameters than a full 3D frequency grid.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Symmetry-Protected Pinch Curves in Classical Spin Liquids arXiv:2607.09470
✗ Mechanism failed 2026

Certainty-Equivalent Auxiliary Critic

For risk-sensitive or recursive objectives, add a separate network that predicts the conditional certainty equivalent of the next-state continuation value, rather than forcing the value network to approximate a nested nonlinear expectation directly. Train the value, policy, and certainty-equivalent heads with Bellman and first-order residuals jointly.

Useful6/10
Difficulty4/10
Novelty6/10
Paper: Deep Learning for Dynamic Programming with Recursive Utility Using First-order Conditions arXiv:2607.09461
✗ Mechanism failed 2026

Resolution-adaptive spectral front end

Replace a fixed Fourier or spectral resolution in a neural operator or sequence model with a data-adaptive spectral cutoff. Keep only modes whose estimated signal energy exceeds the noise-amplification and discretization floor implied by the available number of trajectories and samples per trajectory. This should reduce overfitting to high-frequency sensor noise and preserve accuracy when the same model is deployed at different sampling resolutions.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: The Cost of Discretization in Functional Linear Regression: Minimax Rates and Adaptation arXiv:2607.09350
✓✓ Beats tuned baseline 2026

Local Characteristic Residual Gating

Transform local neural residuals into the Ripa model's characteristic coordinates before spatial aggregation, apply a mode-dependent gate based on neighboring characteristic jumps, and transform back. This lets the model damp oscillatory acoustic or equilibrium-mode corrections near discontinuities without globally smoothing every feature.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Fifth-Order Well-Balanced Path-Conservative A-WENO Scheme for the Ripa Model arXiv:2607.09293
✗ Mechanism failed 2026

Resolvent Fractional-Power Layer

Parameterize a learned feature-space operator as accretive but not necessarily symmetric, then apply its fractional power through a finite positive mixture of shifted resolvents. This provides a matrix-function layer that can represent directional and rotational interactions while avoiding unstable eigendecomposition of nonnormal matrices.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Functions and Means of Accretive Operators arXiv:2607.09152
✗ Mechanism failed 2026

Thermodynamic Two-State Expert Gate

Add a slow latent two-state gate to a recurrent, state-space, or world-model network so that separate experts represent two qualitatively different dynamical regimes. Train the gate using the paper's two-state population and fluctuation mechanism rather than allowing an unconstrained softmax to average incompatible regimes. The model should allocate extra capacity near the gate's susceptibility peak, where regime uncertainty and forecast variance are predicted to be largest.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Structural Origin of Water Heat Capacity Anomaly from Classical and Quantum Simulations arXiv:2607.08957
△ Mechanism confirmed, baseline not beaten 2026

Cross-Ratio Reversible Lattice Layer

Represent a hidden state as complex-valued points on a two-dimensional lattice and replace unconstrained local updates by the exact harmonic-quadrilateral completion rule from discrete conformal geometry. Given three corners of a plaquette, compute the fourth corner by a Mobius-rational formula enforcing cross-ratio minus one, then use a learned readout or forcing term for task-specific predictions. The layer supplies a hard geometric inductive bias and a directly measurable local constraint…

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Integrability of Cauchy problems for discrete conformal maps and circle patterns arXiv:2607.08901
△ Mechanism confirmed, baseline not beaten 2026

Cross-Channel Scattering Front End

Add a differentiable SNST layer before an EEG classifier or sequence model. For every local channel pair and wavelet band, compute the magnitude of the complex cross-channel analytic response, then average it over a controllable temporal window and concatenate it with ordinary channelwise features. This gives the model an explicit, phase-robust amplitude-coupling representation that is especially useful when labeled training data are scarce.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: Spatial Neighboring Scattering Transform: A Cross-Channel Amplitude Coupling Measure for EEG Connectivity arXiv:2607.08855
△ Mechanism confirmed, baseline not beaten 2026

Connectivity-Preserving Wedge Token Pooling

Replace a large graph-token set by a smaller set of connected wedge regions generated through adaptive two-seed shortest-path partitions. Each pooled token is the mean of the node features in its region, while the binary partition tree and region sizes are retained for unpooling or skip connections. This provides a deterministic, graph-aware alternative to arbitrary token merging that can be inserted before graph-transformer message passing.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Tonnetz-Driven Graph Wedgelet for Harmonic Complexity Reduction in Music Scores arXiv:2607.08806
✗ Failed on benchmark 2026

Task-Visible Axis Pruning

Prune hidden units only after testing whether their nonlinear gate is task-visible and downstream-used. For ReLU, a unit is removable when its preactivation does not cross zero on the task patch or its outgoing weight column is zero; this is a more structural criterion than weight magnitude and can be applied during width search or post-training compression.

Useful6/10
Difficulty3/10
Novelty5/10
Paper: Contravariance Theory: Strong Alignment for Minimal Solutions to Hard Tasks arXiv:2607.08561
✗ Failed on benchmark 2026

Wide-tree invariant alignment layer

Replace a purely pairwise embedding similarity used for set alignment with a sum of rooted-tree contraction scores. Each tree feature aggregates products of several coordinate-level interactions and can preserve correspondence information under an unknown orthogonal transformation, allowing matching from moderate correlation rather than nearly identical embeddings. Use the resulting score matrix for Hungarian matching, contrastive loss, or a differentiable Sinkhorn assignment.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: High-Dimensional Procrustes Matching via Tree Counts arXiv:2607.08538
✓✓ Beats tuned baseline 2026

Finite-group relative message passing

Use the quotient group's generator classes as a finite relation vocabulary and tie message functions by group displacement instead of by individual graph edges. This creates a compact, exactly consistent relation-aware GNN that can recognize repeated local structure and transfer parameters across graph instances sharing the same Cayley geometry.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Minimal Isometric Embeddings of Graphs into Cayley Graphs of Finite Abelian Groups arXiv:2607.07920
✗ Mechanism failed 2026

Innovation-Compensated Latent Policy

In a partially observed reinforcement-learning or model-based control agent, expose the state-estimator innovation to the action head through a dedicated residual feedback branch. The policy produces a nominal action from the estimated latent state, while a learned innovation-compensation branch corrects actions when observations disagree with predicted latent dynamics. This explicitly separates nominal policy behavior from estimation-induced corrections and should help during fast transients…

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Revisiting Certainty Equivalence: The Structural Coupling Between Estimation and Control in Underactuated Nonlinear Systems arXiv:2607.07276
✗ Failed on benchmark 2026

Age-conditioned semi-Markov router

Augment a neural router with the age of its current expert or latent regime and use an age-dependent hazard to determine when switching is likely. Unlike ordinary token-wise softmax routing, the router can learn non-geometric residence times, suppressing unstable expert oscillations while still allowing rapid transitions when the current regime becomes inappropriate.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Semi-Markovian switching in a fluctuating harmonic trap: An age-structured formulation arXiv:2607.05173
△ Mechanism confirmed, baseline not beaten 2026

Spectral Hamiltonian Neuron

Replace a scalar neuron activation with a matrix function of a learned Hamiltonian. Fixed Hermitian interaction operators are combined as a trainable linear Hamiltonian, the activation is applied to its eigenvalues, and the resulting observable is measured on an input quantum state. Noncommuting interaction terms provide a controlled source of expressivity beyond an ordinary scalar neuron.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Canonical quantization of neurons arXiv:2607.05000
△ Mechanism confirmed, baseline not beaten 2026

Bilinear Input-Conditioned Koopman Cell

Replace an unconstrained input-conditioned recurrent transition with a bilinear latent update, so controls modulate a fixed linear latent dynamics matrix through low-rank state-input interactions. The resulting cell preserves the computational simplicity of linear propagation while representing multiplicative effects of actions that an additive control term cannot capture efficiently.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Koopman operator theory: fundamentals, control, and applications arXiv:2607.01819
△ Mechanism confirmed, baseline not beaten 2026

Bounded Commuting Cochain Layer

Replace independently predicted node, edge, and face features on a simplicial mesh by a coupled projection layer that is idempotent, bounded in a mass-matrix norm, and approximately commutes with the discrete exterior derivative. The layer can be inserted after an ordinary graph-neural update and should suppress topologically inconsistent feature components without requiring the downstream network to learn these constraints from data.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: An Approximate Bounded Cochain Projection arXiv:2607.07457
△ Mechanism confirmed, baseline not beaten 2026

Binary-form symmetric-power equivariant layer

Replace an unconstrained feature vector of size n+1 by the coefficients of a homogeneous degree-n binary polynomial and make the layer transform through the irreducible symmetric-power representation of GL_2(R). For n=4 this creates a five-channel equivariant feature block whose transformation law is exact rather than learned through augmentation.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: On 4-dimensional convex projective domains invariant by a lattice of $\mathrm{SL}_2 (\mathbb{R})$ arXiv:2607.07150
✗ Failed on benchmark 2026

Hankel Residual Observer

Attach a model-free residual-dynamics observer to a neural multi-step forecaster. Instead of asking the network to relearn persistent periodic or autoregressive disturbances, maintain a Hankel dictionary of recent forecast errors and use ridge reconstruction to predict the next residual sequence online. Add the predicted residual to the network forecast with a confidence-dependent correction gain.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: Model-Free Disturbance Observer with Online Modification: Listening to MFDOOM arXiv:2607.07082