Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

△ Mechanism confirmed, baseline not beaten 2026

Causal E/I Micro-Event Cell

Replace signed-input aggregation in a spiking recurrent cell with a causal micro-event queue that processes excitatory and inhibitory arrivals in timestamp order, applying threshold and reset after each event. This preserves computations that disappear when all events in a timestep are replaced by one net current, particularly near threshold.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Order-Sensitive Fast-Synapse Limits in Sparse Excitatory-Inhibitory Threshold-Reset Networks arXiv:2608.16701
△ Mechanism confirmed, baseline not beaten 2026

Weighted Resolvent-Equivariant Attention

Add a weighted reflection symmetry to an attention or graph-propagation matrix instead of requiring ordinary permutation equivariance. For paired positions or graph nodes related by an involution, penalize the failure of the propagation operator to commute with the weighted reflection; this makes all geometric multi-step propagations symmetry-compatible. The method is suitable for data with mirror, reversal, paired-agent, or left/right structure where the two sides have unequal importance…

Useful7/10
Difficulty4/10
Novelty6/10
Paper: Resolvent intertwining and spectral duality in Markov chains with geometric resetting arXiv:2608.15140
△ Mechanism confirmed, baseline not beaten 2026

Persistent Workspace for Online Adaptation

Turn the latent substrate into a persistent computational workspace for sequential inputs: each new observation is written into a designated subspace, processed by the same local rule, decoded, and then selectively retained or reset. This creates a compact recurrent model whose state can accumulate algorithmic information across a stream without expanding the parameter count.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Emergent Models: Intelligence from Tiny Substrates arXiv:2608.14019
✓✓ Beats tuned baseline 2026

Tiny Local Recurrence with Adaptive Computation

Replace a stack of independently parameterized residual or MLP blocks with a small latent grid or vector repeatedly updated by one shared transition rule. Let the number of updates depend on the current latent state, so easy examples terminate early while hard examples receive more computation, potentially improving parameter efficiency and extrapolation.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Emergent Models: Intelligence from Tiny Substrates arXiv:2608.14019
✗ Mechanism failed 2026

Pisot-Orbit Deterministic JL Layer

Replace a dense random projection used before retrieval, classification, or expert routing with a publicly reproducible matrix generated by a Pisot beta-transformation orbit. Search over a small public seed and sampling gap to select one matrix that preserves the calibration set's pairwise distances, then freeze it for training and inference. The projection removes random-matrix storage and makes the same embedding transform exactly reproducible across servers or proof systems.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Deterministic Johnson--Lindenstrauss Projections from Pisot $β$-Transformations for Zero-Knowledge Private Routing arXiv:2608.13078
✗ Failed on benchmark 2026

Sensitivity-Conditioned Neural ODE Pruning

Use trajectory sensitivities to remove neural units or parameter groups whose effects are redundant over the available data support. A parameter group is pruned when its Fisher contribution is small or its sensitivity is nearly collinear with other groups, producing a compact neural ODE without relying only on parameter magnitude.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: Identifiability-aware neural ordinary differential equations for parsimonious and reliable dynamic modelling arXiv:2608.13044
✗ Failed on benchmark 2026

Critical stochastic min-plus tree layer

Replace deterministic binary-tree pooling or hierarchical feature aggregation by a stochastic merge that chooses either elementwise addition or elementwise minimum. The mixing probability p controls whether zero or sparse states proliferate or disappear, with a predicted absorbing-state transition at p = 1/2.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: Finite-depth scaling and an exact Bernoulli-leaf identity for the min-plus process on the binary tree arXiv:2608.12295
△ Mechanism confirmed, baseline not beaten 2026

First-Spike Laplacian Attention

Replace multiplicative query-key attention scores with an affinity based on the l1 distance between first-spike latency vectors. For each query token and key token, small latency differences produce large affinity and distant timings decay exponentially, yielding a locality-sensitive attention pattern naturally compatible with leaky spiking neurons.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Lapis: Laplacian Spiking Attention via First-Spike Timing and Membrane Leakage arXiv:2608.11865
△ Mechanism confirmed, baseline not beaten 2026

Intrinsic-Rank Filter Memory for Actor-Critic

Replace an oversized recurrent hidden state or raw history stack with a causal filtered input-output lift followed by an SVD-selected bottleneck. The actor, critic, and Bellman regression operate only on the identifiable memory coordinates, preventing deterministic null directions from being fitted as if they were independent state variables.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Reinforcement Learning-Based Output Feedback LQR for Continuous-Time MIMO Systems arXiv:2608.11750
△ Mechanism confirmed, baseline not beaten 2026

Congestion-aware equimarginal MoE router

Replace independent token-to-expert softmax routing with a fixed-budget congestion game. Each token group distributes a fixed routing mass across experts, while the marginal value of an expert decreases as other groups send mass there. Iteratively route toward the highest current marginal utility and exploit sorted-prefix supports to produce sparse, capacity-aware assignments.

Useful7/10
Difficulty5/10
Novelty5/10
Paper: The Game of Marginal Utilities arXiv:2608.10373
✗ Failed on benchmark 2026

Følner-Gated Message Passing

Add an online receptive-field expansion monitor to a graph neural network and use it to gate message-passing depth or invoke graph pooling. For a sampled node set F and propagation neighborhood K, continue fine-scale propagation only while the growth ratio |KF|/|F| is close to one; when it is persistently expansive, replace further propagation with pooling, local attention, or long-range skip messages. This transfers the paper's Følner-versus-paradoxical mechanism into an architecture-level…

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Fiberwise amenability of étale groupoids arXiv:2608.09796
△ Mechanism confirmed, baseline not beaten 2026

GECC-Gated Loop-Aware Message Passing

Construct order-n generalized edges from intersections of local ego-subgraphs and use their overlap statistics to correct ordinary one-hop aggregation. A learned gate should activate the correction only when local generalized-edge closure is high, because dense but internally inconsistent overlaps are precisely where naive loop corrections can become unreliable.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Ensemble-level loopy message passing with generalized-edge closure for percolation arXiv:2608.09397
△ Mechanism confirmed, baseline not beaten 2026

Hodge-Selective Edge Dynamics

Replace an unconstrained edge-feature residual update in a graph neural network with separate cut-space and harmonic-space updates. The cut branch carries transfer information visible at nodes, while the harmonic branch models cycle circulation and can be given an independently chosen contraction rate, preventing persistent or unstable circulation features from contaminating node predictions.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: From a Scalar Parabolic Oscillator to Topological Thermostats: Selective Feeback Control of Harmonic Flow Modes arXiv:2608.07768
△ Mechanism confirmed, baseline not beaten 2026

Spectral-Gap Synchronizing Neural Graph Dynamics

Build a graph neural dynamical system whose node states are coupled through a graph Laplacian, using the Laplacian spectral gap as a controllable synchronization mechanism. Increasing coupling strength or algebraic connectivity should selectively suppress disagreement modes, producing a measurable faster decay of node-to-node errors without requiring stronger contraction of the common mode.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Contraction Analysis of Holomorphic Dynamical Systems via the Intrinsic Kobayashi Metric arXiv:2608.07551
△ Mechanism confirmed, baseline not beaten 2026

Maximum-cardinality token-expert routing

Construct a bipartite candidate graph between tokens and experts from the router’s top-k logits, then solve a capacity-constrained maximum-cardinality matching rather than dispatching each token independently. The mechanism targets the extreme tail of routing completion: it should reduce unmatched or repeatedly reassigned tokens and lower maximum dispatch delay and expert starvation, even when average routing quality changes little.

Useful7/10
Difficulty6/10
Novelty4/10
Paper: Collective search-and-capture under competing assignment policies arXiv:2608.06084
△ Mechanism confirmed, baseline not beaten 2026

Spectral Message Basis

Replace full agent-to-agent state transmission with coefficients in a learned dominant Koopman mode basis. Agents communicate only the leading spectral coordinates that explain slowly decaying collective behavior, while retaining a certificate based on the spectral gap and subdominant eigenvalue to decide whether the compressed representation is safe.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Certifying Collective Reasoning in Multi-Agent Systems via Koopman Spectral Analysis arXiv:2608.05956
△ Mechanism confirmed, baseline not beaten 2026

Toda-Krylov adaptive polynomial layer

Replace a fixed-order polynomial or recurrent state-space block by an Arnoldi basis built from a learned operator and the current input, and use subdiagonal coefficients as geometry-aware gates over Krylov depth. The gates quantify how much genuinely new direction each operator application contributes, allowing the layer to stop early near Krylov breakdown and suppress redundant or unstable directions.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Two-dimensional Toda--Arnoldi correspondence: Holomorphic Krylov geometry and counterdiabatic transport arXiv:2608.04850
△ Mechanism confirmed, baseline not beaten 2026

Explicit Multi-Timescale Memory Bank

Use a bank of damped rotational state channels with a deliberately spread decay spectrum, allowing one recurrent layer to represent short, medium, and long temporal dependencies without relying on a single learned spectral radius. Concatenate the channels and train a readout or downstream nonlinear head to select the appropriate memory timescale.

Useful7/10
Difficulty4/10
Novelty7/10
Paper: Lindblad-Inspired Multi-Timescale Reservoir Computing with Separable Rotation and Dissipation arXiv:2608.04028
△ Mechanism confirmed, baseline not beaten 2026

Normal Damped-Rotation Recurrent Layer

Replace an unconstrained recurrent matrix with an orthogonally mixed block diagonal matrix whose blocks are independently parameterized damped rotations. The model receives explicit phase mixing from the rotation frequencies and controlled forgetting from the decay rates, while its linear recurrent dynamics have a known contraction factor before the nonlinear activation.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Lindblad-Inspired Multi-Timescale Reservoir Computing with Separable Rotation and Dissipation arXiv:2608.04028
✗ Failed on benchmark 2026

Chemotactic Feature Transport Layer

Replace part of a CNN or continuous-depth feature block with two coupled feature fields. One field is transported up gradients of the other through a conservative cross-gradient flux, creating adaptive spatial organization that ordinary diffusion or symmetric convolution cannot produce. The coupling strength and dominant wavelength are controlled by a directly testable linear-instability boundary.

Useful7/10
Difficulty6/10
Novelty8/10
Paper: Pattern formation: reactivity is not necessary for chemotaxis--driven instabilities arXiv:2608.03685
✓✓ Beats tuned baseline 2026

Two-Channel Fractal Renormalization Network

Replace ordinary depth-wise feature propagation by a ternary hierarchical block that recursively aggregates three child representations while maintaining separate neutral and defect channels. The block is initialized from the Sierpinski six-vertex recursion, then optionally learns a bounded correction. The neutral channel preserves the paper's cubic mixing law, while the defect channel provides a controlled route for long-range and nonlocal interactions.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Fractal deconfinement and confinement in Sierpinski ice arXiv:2608.02741
✓✓ Beats tuned baseline 2026

Parallel Quadratic Tree Layer

Replace sequential upward message passing in a tree-structured neural module with rake–compress contraction of quadratic latent-state messages. Each node stores a quadratic value function and each edge stores a linear transition or coupling triple; leaf elimination and unary-node compression are implemented as batched Schur complements, followed by a reverse pass that reconstructs node latents and edge outputs. The layer is exact for Gaussian or quadratic latent models and remains…

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Rake-Compress Riccati Recursions for Parallel Scenario-Tree Model Predictive Control arXiv:2608.01332
△ Mechanism confirmed, baseline not beaten 2026

Isometric tensor-network token mixer

Use the relaxed QFT tensor-network topology as a trainable norm-preserving mixer inside a neural block, replacing a dense token-mixing matrix or an expensive global convolution. The network learns data-adapted global interactions while retaining structured O(N log^2 N) application and an exact cheap inverse, making it suitable for image tokens, long sequences, or reversible residual blocks.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Fast Trainable Multilinear Bases for Image Compression arXiv:2608.00053
✓✓ Beats tuned baseline 2026

Symmetry-Preserving Flow Layer

Construct hidden dynamics from permutation-equivariant vector fields and impose antisymmetry through an explicit antisymmetrizing readout. This prevents optimization from learning multiple equivalent copies of the same configuration and makes forbidden symmetry violations exactly zero, rather than merely penalizing them. The design applies to set models, particle systems, graph networks, and architectures handling unordered tokens.

Useful7/10
Difficulty5/10
Novelty4/10
Paper: Spindrift: Learning quantum degeneracy from thermal purity in restricted path integral Monte Carlo arXiv:2607.29590