Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✗ Failed on benchmark 2026

Small-gain certified modular network

Partition a neural network into independently trained or independently monitored modules and constrain their cross-module interaction gain using a compositional contraction certificate. This enables stable deep modular MLPs, graph blocks, or recurrent modules without estimating the full network Jacobian, while providing an explicit coupling threshold for when the architecture loses contraction.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Contraction Certification from Streaming Data: Wasserstein Robustness and Compositional Stability for Interconnected Nonlinear System arXiv:2607.11982
△ Mechanism confirmed, baseline not beaten 2026

Certified contraction implicit layer

Replace a deep feed-forward block by the fixed point z=phi(Wz+Vx+b), with the recurrent weight W constrained so that the fixed point is unique for every input. The same condition makes forward fixed-point iteration stable and makes implicit differentiation well-conditioned, allowing depth-independent memory usage while providing a measurable spectral failure boundary.

Useful8/10
Difficulty5/10
Novelty4/10
Paper: Implicit Neural Networks as Static Controllers: Certificates and Performance Separation arXiv:2607.11122
✗ Failed on benchmark 2026

Topology-Aware Streaming Jacobian Monitor

For a recurrent or graph neural network with known local connectivity, estimate each node's local Jacobian row using only graph neighbors rather than all hidden coordinates. Use the resulting sparse Jacobian both to compute a contraction certificate and to regularize training toward dynamically local interactions, reducing estimator variance and the number of samples required for reliable stability decisions.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Streaming Contraction Certificates for Nonlinear Networks: Topology-Aware Data Sufficiency with Partial Observation arXiv:2607.10893
△ Mechanism confirmed, baseline not beaten 2026

Certified Bi-Lipschitz Recurrent Cell

Replace an unconstrained RNN or state-space layer with an implicit recurrent cell whose nonlinear algebraic loop is well posed and whose forward dynamics are contracting and strongly input-output monotone. The same certificate guarantees a causal inverse with bounded gain, so sequence predictions should be insensitive to initial-state mismatch while remaining responsive to input perturbations.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Robustly Invertible Nonlinear Dynamics and the BiLipREN: From Inversion-Based Control to Generative Trajectory Modelling arXiv:2607.10026
✗ Failed on benchmark 2026

Balanced State-Order Compression

Compress each hidden layer by retaining directions that are simultaneously reachable from the observed input distribution and observable at the network output. Unlike PCA or SVD, the retained subspace is weighted by downstream task sensitivity, so high-variance but output-irrelevant directions can be removed while low-variance predictive directions are preserved.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Empirical Minimal-Realisation Compression of Deep Neural Networks via Controllability-Observability Tests arXiv:2607.05457
△ Mechanism confirmed, baseline not beaten 2026

Degree-Calibrated Stable Residual Flow

Construct a continuous-depth or recurrent residual block with a prescribed polynomial Lyapunov decay near its equilibrium. The architecture combines a fixed radial stabilizer with a learned component that is constrained to have zero radial projection, allowing slow algebraic transients and long memory while preventing asymptotic hidden-state growth.

Useful7/10
Difficulty6/10
Novelty8/10
Paper: On a Gradation for Asymptotic Stability arXiv:2609.03120
✗ Failed on benchmark 2026

Positive-real rational resolvent mixer

Replace an unconstrained token mixer or feed-forward residual map with a feature-conditioned rational operator whose transfer matrix is positive real on the positive orthant. Apply it through a resolvent, rather than an additive residual, so that the accretivity certificate yields a nonexpansive implicit update and suppresses activation explosions.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: The classes of bivariate Schur and Herglotz matrix-valued rational functions: realizations, symmetrizations, and related determinantal representations arXiv:2609.03054
✓✓ Beats tuned baseline 2026

Spectral pinning of neural modules

Represent communicating layers, experts, or distributed workers as nodes of a weighted graph and apply strong corrective updates only to a small pinned subset. Select pins by the increase they produce in the grounded Laplacian smallest eigenvalue, because this spectral gap predicts the decay rate of representation disagreement.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Event-Triggered Pinning Impulsive Control of Complex Networks with Actuation Delays: Stability Analysis and Zeno-Free Conditions arXiv:2608.24074
△ Mechanism confirmed, baseline not beaten 2026

Positive-Regime Observable ReLU State Space

Constrain recurrent preactivations to remain nonnegative so that ReLU acts as the identity along realized trajectories. The hidden dynamics then admit a classical linear observability matrix, allowing principled hidden-coordinate selection and conditioning control instead of relying on potentially destructive activation masks.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: On the Number of Observation Nodes in Recurrent Neural Networks with Linear Threshold and ReLU Functions arXiv:2608.29650
△ Mechanism confirmed, baseline not beaten 2026

Monotone Compositional Reachability Critic

Train separate neural value functions for primitive reachability, avoidance, or target-reaching tasks, then combine them with a coordinatewise monotone aggregator whose derivatives with respect to all primitive values are nonnegative. This transfers the paper's exact two-player decomposition condition into a modular critic architecture: adding a new target changes only one primitive critic and the aggregator, rather than requiring a new high-dimensional value function.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Exact Decomposition of Value Functions for Two-Player Games in Hamilton-Jacobi Reachability arXiv:2608.27654
✗ Failed on benchmark 2026

Quadratic center-manifold bottleneck

Replace a conventional linear decoder in an autoencoder or latent state-space model with an explicit quadratic manifold decoder, allowing a small latent vector to represent curved and transport-like state trajectories. Add a dynamics-aware invariance loss that penalizes the discrepancy between the time derivative of the quadratic manifold and the neural dynamics evaluated on that manifold.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Beyond linear subspaces: Nonlinear moment matching meets quadratic manifolds arXiv:2608.19486
✗ Mechanism failed 2026

Automaton-Supervised Neural Policy Shield

Attach a deterministic supervisory automaton to a neural policy or sequence model and mask every event disabled by the current supervisor state. Use a short receding-horizon planner over admissible events to resolve conflicts between neural preferences and shared-resource constraints. The network scores useful actions, while the automaton supplies an exact safety layer.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Model Predictive Supervisory Control for Hierarchical and Distributed UAS Traffic Management arXiv:2608.18353
△ Mechanism confirmed, baseline not beaten 2026

Koopman Hankel Dual Autoencoder

Replace pointwise sequence reconstruction with reconstruction of overlapping past and future Hankel windows in a shared latent manifold. A first encoder compresses the delay-coordinate trajectory, while a second decoder or predictor reconstructs the future block from the latent state; training therefore penalizes representations that fit observations but do not preserve dynamical evolution.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Fault detection on manifolds of nonlinear dynamical systems with dual autoencoders arXiv:2608.17698
△ Mechanism confirmed, baseline not beaten 2026

Intrinsic-Rank Filter Memory for Actor-Critic

Replace an oversized recurrent hidden state or raw history stack with a causal filtered input-output lift followed by an SVD-selected bottleneck. The actor, critic, and Bellman regression operate only on the identifiable memory coordinates, preventing deterministic null directions from being fitted as if they were independent state variables.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Reinforcement Learning-Based Output Feedback LQR for Continuous-Time MIMO Systems arXiv:2608.11750
△ Mechanism confirmed, baseline not beaten 2026

Finite Hyperplane Representative Verification

Replace dense continuous action search during neural-controller verification with a finite set of representative inputs induced by affine pieces of the interval neural dynamics. This makes safety checking parallel over state cells and candidate actions, enabling much cheaper certification or repeated safe-set updates.

Useful7/10
Difficulty7/10
Novelty8/10
Paper: Computing the Maximal Controlled Invariant Set for Neural Network Control Systems arXiv:2608.07908
△ Mechanism confirmed, baseline not beaten 2026

Hodge-Selective Edge Dynamics

Replace an unconstrained edge-feature residual update in a graph neural network with separate cut-space and harmonic-space updates. The cut branch carries transfer information visible at nodes, while the harmonic branch models cycle circulation and can be given an independently chosen contraction rate, preventing persistent or unstable circulation features from contaminating node predictions.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: From a Scalar Parabolic Oscillator to Topological Thermostats: Selective Feeback Control of Harmonic Flow Modes arXiv:2608.07768
✗ Failed on benchmark 2026

Information-Gated Attention

Use predicted covariance reduction as a differentiable gate for selecting tokens, views, sensors, or retrieved demonstrations. The gate favors inputs with high expected information gain while accounting for acquisition cost, turning attention and data collection into active observability optimization.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Information-Aware Model Predictive Control for Satellite Inspection arXiv:2608.07765
△ Mechanism confirmed, baseline not beaten 2026

Spectral-Gap Synchronizing Neural Graph Dynamics

Build a graph neural dynamical system whose node states are coupled through a graph Laplacian, using the Laplacian spectral gap as a controllable synchronization mechanism. Increasing coupling strength or algebraic connectivity should selectively suppress disagreement modes, producing a measurable faster decay of node-to-node errors without requiring stronger contraction of the common mode.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Contraction Analysis of Holomorphic Dynamical Systems via the Intrinsic Kobayashi Metric arXiv:2608.07551
✗ Failed on benchmark 2026

Matching-Controllable Recurrent State Space

Construct the sparse transition matrix and input projection of a recurrent or state-space layer so that every hidden-state row is covered by a matching in the controllability core. This prevents hidden directions from becoming unreachable from the input sequence, especially in multi-input systems and across a distribution of transition matrices or task conditions.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Structural Averaged Controllability for Linear Ensemble Systems: Multi-input Case arXiv:2607.27706
△ Mechanism confirmed, baseline not beaten 2026

Non-Gaussian Perron–Frobenius Latent Filter

Replace Gaussian covariance propagation in a neural state-space model with a finite Perron–Frobenius operator acting on coefficients of a learned density basis. A neural encoder maps observations to latent states, while an eDMD-derived matrix transports the full coefficient vector and supports multimodal or skewed uncertainty. This creates a cheap deterministic uncertainty layer that can be rolled forward for long horizons without repeatedly sampling particles.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: An extended Perron-Frobenius operator filter for nonlinear state estimation arXiv:2607.26632
✗ Mechanism failed 2026

Autocatalytic Hysteresis Memory Cell

Replace or augment a recurrent hidden coordinate with a nonnegative bistable autocatalytic state driven by an external control signal. The cell retains information through metastable low and high states, while a periodic or slowly varying control produces a controlled phase lag and hysteresis useful for temporal regime detection. Explicit noise can be injected to test whether it enhances switching near the predicted intermediate-frequency regime.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Dynamic hysteresis in an autocatalytic reaction network arXiv:2607.24163
△ Mechanism confirmed, baseline not beaten 2026

Feasible Action Mapping Safety Layer

Let a neural policy emit an unconstrained abstract action z, then solve a state-dependent feasibility problem that maps z to an admissible optimal-control parameter p before execution. Unlike coordinate-wise clipping, the mapping accounts for predicted dynamics, coupled state and input constraints, and recursive feasibility, allowing the policy to retain a simple unconstrained output space while the controller enforces plant constraints.

Useful7/10
Difficulty6/10
Novelty5/10
Paper: Bridging Reinforcement Learning and Optimal Control via Feasible Action Mapping arXiv:2607.23930
△ Mechanism confirmed, baseline not beaten 2026

Petri-Net Safety Shield for Neural Policies

Compile Boolean safety requirements over a discrete plant state into a Petri Net and place a transition mask between a neural policy and the environment. The policy may score all candidate actions, but only transitions enabled at the current marking are executable; this prevents invariant-violating actions and exposes deadlocks or unreachable policy proposals explicitly.

Useful7/10
Difficulty5/10
Novelty5/10
Paper: Constraint-Driven Synthesis of Hyper Petri Nets arXiv:2607.22062
△ Mechanism confirmed, baseline not beaten 2026

Tangent-Branch Neural Evasion Layer

Wrap a neural multi-agent policy with an analytic planner that generates turn-straight trajectories tangent to pursuer surveillance disks, then selects the branch with the smallest predicted completion time. The network supplies high-level preferences or residual corrections, while the geometric layer prevents unnecessarily entering exclusion regions and exposes an explicit branch-switching signal for training.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Semi-Explicit Solutions to the Prying-Pedestrian Surveillance-Evasion Differential Game and Extensions to Two Pursuers arXiv:2607.21087