Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

Unverified 2026

Probability-Preserving Zonotopic Neural Uncertainty

Attach a finite mixture of zonotopes to each uncertain neural input or hidden state, and propagate every mixture component through affine layers and conservative nonlinear relaxations. When the number of components grows, merge components only with an enclosing zonotope and sum their probability masses, preserving a formal lower bound on the probability that the true activation lies in the represented set.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: The Zonotopic Mixture Filter arXiv:2608.17897
Unverified 2026

Cactus-Graph Phase Budgeting

Build neural computation graphs with explicitly phase-budgeted serial and parallel branches, treating serial compositions as SRG products and parallel residual branches as SRG sums. Allocate phase centers theta_i so that every loop or branch aggregate stays away from -1, enabling stability-aware architecture search and constructive control of branch gains.

Useful6/10
Difficulty7/10
Novelty8/10
Paper: The $θ$-Symmetric SRG with Applications to Stability of Cactus Dynamic Networks arXiv:2608.12591
Unverified 2026

Single-Node Observable Leaky-RNN

Construct a sparse recurrent network with positive edge weights and Leaky-ReLU updates so that one selected hidden node, observed over a finite time window, contains enough information to reconstruct the full hidden state. Add an auxiliary decoder from the observed trajectory to the initial state or current state, and use graph rewiring or edge-growth until every hidden node has a directed path to the sensor within the observation horizon.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: On the Observability and Controllability of Leaky-ReLU Networks arXiv:2608.09059
Unverified 2026

Invertible Fourier Surrogate for Periodic Sequence Modeling

Represent periodic input-output behavior using a compact real vector of Fourier coefficients and learn an invertible neural map from input coefficients to output coefficients. Inference then obtains the input representation for a desired periodic output by a single inverse pass instead of iterative optimization through a nonlinear forward model, while the Fourier representation reduces sequence dimensionality when high-rate signals are spectrally sparse.

Useful6/10
Difficulty6/10
Novelty4/10
Paper: Certified Feedforward Tracking for Unknown Nonlinear Systems via Invertible Neural Networks arXiv:2608.06419
Unverified 2026

KAM-Stabilized Quasiperiodic Recurrent Memory

Construct a recurrent module with a phase variable and a transverse memory coordinate modeled on a perturbed twist map. Train the transverse state to lie on an invariant graph over the phase, while the phase follows an approximately irrational rigid rotation. A KAM-inspired graph correction and residual penalty should reduce long-horizon drift in recurrent prediction.

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Persistence of invariant graphs for twist maps under analytic perturbations arXiv:2608.05239
Unverified 2026

Noncommutative controllability regularizer

Equip a recurrent or state-space layer with multiple noncommuting transition operators and regularize the span of finite operator words applied to the input injection matrix. This discourages hidden directions that cannot be reached from the input and may improve long-range input influence, gradient propagation, and robustness under operator switching.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: A noncommunicative Kalman condition for null controllability of backward stochastic parabolic systems arXiv:2608.01836
Unverified 2026

Normalized Scheduling-Degree Truncation

Use normalized scheduling variables and explicitly cap the degree of their products in a neural LPV or mixture-of-dynamics model. Instead of allowing every multiplicative interaction between scheduling coordinates and past or future features, retain only monomials below a chosen degree threshold. This produces a controllable approximation knob between a purely linear model and a full lifted predictor, while avoiding unstable extrapolation caused by poorly scaled high-degree features.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: A subspace approach to data-driven predictive control for linear parameter-varying systems arXiv:2607.28490
Unverified 2026

Layered Structural Reachability for Neural States

Treat the hidden-state Jacobian of an RNN, SSM, or graph neural network as a directed matrix-weighted network and decompose repeated block couplings into scalar interaction layers. Use layer-specific structural controllability to select input, skip, reset, or readout channels that can reach all hidden dimensions, and reject architectures with structurally unreachable states before training.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: On the Strong Structural Controllability of Matrix-Weighted Networks arXiv:2607.27852
Unverified 2026

Low-Order Robust Functional Observer

Attach a small dynamical observer to a neural ODE, RNN, or state-space model and make it estimate only a task-relevant functional of the hidden state, such as logits, value features, or control-relevant projections. Use an incremental quadratic constraint and a bounded-real penalty to make the observer robust to hidden-state nonlinearities and input disturbances, instead of reconstructing the full latent state.

Useful6/10
Difficulty6/10
Novelty5/10
Paper: Functional H_infinity Filtering for Descriptor Systems with Incrementally Quadratic Nonlinearities under Disturbances arXiv:2607.25000
Unverified 2026

Connectivity-aware ATC gradient tracking

Use the paper's mode decomposition to choose between ordinary DIGing and augmented ATC-DIGing/AugDGM according to the communication spectrum and curvature. The augmented scheme changes the disagreement feedback gain mode by mode, which can reduce the dominant pole radius on well-connected graphs without changing the neural-network architecture.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Exact Worst-case Convergence Rates of Distributed Gradient Tracking Methods arXiv:2607.23601
Unverified 2026

Imaginary-Axis Gramian Compression for Neural SSMs

Replace a large stable linear state-space or recurrent layer by a lower-order balanced realization computed from frequency-targeted controllability and observability Gramians. Use generalized low-rank ADI with imaginary-axis shifts concentrated at frequencies that dominate the training data, then retain states associated with the largest approximate Hankel singular values. This should reduce recurrent inference cost while preserving the layer's input-output response in the selected frequency…

Useful6/10
Difficulty6/10
Novelty7/10
Paper: A New Low-Rank Cholesky-Factor ADI Algorithm Allowing Shifts Anywhere in the Complex Plane with Applications to Data-Driven Model Reduction arXiv:2607.21969
Unverified 2026

Linear-solve ensemble controller

Add a shallow neural interpolation controller to a neural ODE or state-space model so one shared vector field matches prescribed derivatives at several anchor trajectories. At every control time, compute controller weights from a small linear system instead of learning all task-specific parameters by backpropagation.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Exact ensemble controllability for neural differential equations via neural interpolation arXiv:2607.21112
Unverified 2026

Deadline-Aware Fair-to-Greedy Router

Use deadline objectives to train or control a router that explicitly trades off completion probability against completed work by a fixed horizon. Begin with fair allocation for robust exploration, then anneal toward a feedback-greedy rule once per-item difficulty estimates have sufficient evidence.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Meeting Uncertain Threats with Feedback arXiv:2607.13648
Unverified 2026

Swarmalator Token Organizer

Augment each token or graph node with a periodic latent position x_i and phase θ_i, then evolve these variables before attention or message passing. Tokens with similar phase attract in x, while tokens with similar position synchronize in θ, producing self-organized groups without an externally specified clustering objective. The coupling strengths J and K provide interpretable controls for aggregation and synchronization, and their sweep should expose the paper's four collective regimes and…

Useful6/10
Difficulty5/10
Novelty8/10
Paper: A solvable normal form for coupled swarmalators arXiv:2607.09810
✗ Mechanism failed 2026

Innovation-Compensated Latent Policy

In a partially observed reinforcement-learning or model-based control agent, expose the state-estimator innovation to the action head through a dedicated residual feedback branch. The policy produces a nominal action from the estimated latent state, while a learned innovation-compensation branch corrects actions when observations disagree with predicted latent dynamics. This explicitly separates nominal policy behavior from estimation-induced corrections and should help during fast transients…

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Revisiting Certainty Equivalence: The Structural Coupling Between Estimation and Control in Underactuated Nonlinear Systems arXiv:2607.07276
△ Mechanism confirmed, baseline not beaten 2026

Bilinear Input-Conditioned Koopman Cell

Replace an unconstrained input-conditioned recurrent transition with a bilinear latent update, so controls modulate a fixed linear latent dynamics matrix through low-rank state-input interactions. The resulting cell preserves the computational simplicity of linear propagation while representing multiplicative effects of actions that an additive control term cannot capture efficiently.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Koopman operator theory: fundamentals, control, and applications arXiv:2607.01819
✗ Failed on benchmark 2026

Hankel Residual Observer

Attach a model-free residual-dynamics observer to a neural multi-step forecaster. Instead of asking the network to relearn persistent periodic or autoregressive disturbances, maintain a Hankel dictionary of recent forecast errors and use ridge reconstruction to predict the next residual sequence online. Add the predicted residual to the network forecast with a confidence-dependent correction gain.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: Model-Free Disturbance Observer with Online Modification: Listening to MFDOOM arXiv:2607.07082
Unverified 2026

Companion Observer Memory for Neural Policies

Replace an unrestricted GRU or attention-based history encoder with a fixed companion-form shift register driven by the current action and observation, followed by a learned nonlinear policy. The register stores a structured finite history, while a learned matrix or MLP readout maps that history to a control-relevant latent state. This should provide a cheaper and more interpretable memory mechanism for partially observed environments, especially when the relevant dynamics are approximately…

Useful5/10
Difficulty4/10
Novelty6/10
Paper: Data-Driven Output Feedback based Analysis and Control for Unknown Discrete-Time Linear System arXiv:2608.18452
Unverified 2026

Velocity-Scaled Symbolic Flow Model

Represent a continuous-time neural dynamical system as a symbolic Markov chain over regions together with a positive learned roof function giving the time spent in each region. Weight local reconstruction and prediction errors by the predicted vector-field speed, following the paper's scaled Hölder coding relation, so that the model does not over-penalize arbitrarily small coordinate errors near equilibria. This produces a hybrid latent model with discrete long-range structure and continuous…

Useful5/10
Difficulty6/10
Novelty7/10
Paper: Symbolic dynamics for non-uniformly hyperbolic flows arXiv:2608.14095
Unverified 2026

Homoclinic Symbolic Reservoir

Construct a periodically driven hybrid recurrent state-space model whose vector field is piecewise smooth across learned switching surfaces. Engineer a transverse homoclinic intersection around a hyperbolic recurrent state; the resulting shift-like invariant set provides a controllable symbolic reservoir for sequence prediction and long-horizon generation.

Useful5/10
Difficulty7/10
Novelty7/10
Paper: Homoclinic Theorems for piecewise smooth vector fields arXiv:2607.09618
Unverified 2026

Pole-Tuned Graph Residual Layer

Use the graph Laplacian spectrum to set the mixing and correction coefficients of a two-state graph-propagation block. Balancing the contraction of low-frequency consensus modes against high-frequency disagreement modes may reduce oversmoothing and make deep graph-neural networks less sensitive to manually selected residual coefficients.

Useful5/10
Difficulty6/10
Novelty5/10
Paper: Optimal Parameter Design for DIGing on Minimizing Unweighted Sum of Squares arXiv:2607.25463