Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

Unverified 2026

PCA-Discovered Implicit Latent Dynamics

Replace an explicit recurrent transition with a learned descriptor relation in latent space, allowing some latent coordinates to satisfy algebraic constraints rather than being numerically integrated. Fit the relation using total-least-squares or iterative PCA on the jointly observed trajectory, so noise in every channel is treated symmetrically and the model can discover whether the latent system is index-0 or index-1.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Subspace Based Identification of Errors-in-Variables Linear Descriptor Systems arXiv:2608.30259
Unverified 2026

Cost-Map Finite-Action Head

Replace online enumeration over a finite action set with a classifier or lookup map whose regions directly return the action minimizing a one-step predictive-control cost. For affine dynamics and quadratic tracking loss, exact action regions are separated by pairwise cost boundaries, so the approximation can be audited against exhaustive predictive control rather than treated as an unconstrained policy.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: A Simple and Extremely Efficient Predictive Control for Power Converters arXiv:2608.22416
Unverified 2026

Integrable Elliptic Memory Cell

Replace an unconstrained recurrent transition with block-diagonal planar rotations whose angles are learned or conditioned on a slowly varying context variable. The resulting hidden-state norm and each two-dimensional block energy are exactly invariant in the ideal recurrence, preventing exploding or vanishing recurrent dynamics while retaining phase information over long horizons.

Useful6/10
Difficulty4/10
Novelty4/10
Paper: Outer Contact Billiards arXiv:2608.19393
Unverified 2026

Dimension-calibrated contractive branch attractor

Construct a recurrent or generative network from finitely many contractive branches whose hidden-state attractor has a prescribed similarity dimension. The branch contraction ratios determine the target complexity through the equation sum_i r_i^s = 1, while a separation penalty approximates the open set condition and prevents branch collapse or excessive overlap.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: The Moran--Hutchinson formula in semimetric spaces arXiv:2608.16817
Unverified 2026

Surrogate-guided topology search for nonlinear reservoirs

Search sparse reservoir wiring in graph space rather than repeatedly testing every candidate with its full nonlinear dynamics. Use graph descriptors to predict validation accuracy and nonlinear feature selectivity, then spend exact simulations on candidates with high predicted performance or high surrogate uncertainty.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: Graph-theoretic design of lasing networks for physical vision arXiv:2608.13097
Unverified 2026

Resonance-Gated Triadic Fourier Layer

Replace unconstrained spectral mixing with a three-component triadic interaction whose strength is determined by the quadratic phase mismatch R(xi,xi_1). Near-resonant products receive high weight because their phases remain coherent, while strongly nonresonant products are attenuated. The resonance bandwidth can be fixed from the frequency grid or learned as a positive parameter.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: On solitary wave solutions with two-frequency parameters to the three-component system of quadratic nonlinear Schrödinger equations arXiv:2608.12983
Unverified 2026

Positive Reflected Bellman Layer

Replace an unconstrained spatial aggregation in a neural PDE surrogate or controlled-dynamics model with a fixed-branch expectation layer. Each output is a maximum over controls of a nonnegative weighted average of next-state values, with reflected overshoots attenuated by Robin factors. Increasing any input value therefore cannot decrease the output, giving a hard monotonicity and positivity property instead of relying on a penalty.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: A Positivity-Preserving Expectation Scheme for Hamilton--Jacobi--Bellman Equations with Oblique Robin Boundary Conditions arXiv:2608.11936
Unverified 2026

Finite-Difference Conditional Neural Operator

Replace a tensor-product network over a low-dimensional state and a large distribution embedding with a neural operator that consumes the distribution vector once and outputs values on a finite-difference grid in the low-dimensional state. Train it with the governing PDE residual, explicit boundary residuals, and optional signed shape constraints, allowing the network to preserve numerical structure that a generic MLP would learn only implicitly.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Mastering Stochastic OLG Models in Continuous Time arXiv:2608.11134
Unverified 2026

Measure-Lifted Entropy Amplifier

Replace a deterministic latent state with a probability measure over latent states, represented by particles or weighted prototypes. Apply the learned latent transition to every particle, so one base trajectory map induces a dynamics on distributions; use an entropy-preservation or entropy-growth regularizer to prevent collapse of the ensemble. The mechanism predicts that any positive base-state trajectory entropy can generate unbounded distinguishability in the ideal measure space through…

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Entropies of compact subsets and supported measures arXiv:2608.09702
Unverified 2026

Nielsen Quaternionic Hyperbolic Latent Layer

Represent each recurrent latent state as a pair of unit quaternions \((q_1,q_2)\in\mathrm{SU}(2)^2\), and evolve it with a composition of elementary Nielsen maps corresponding to a chosen hyperbolic matrix \(A\in\mathrm{SL}(2,\mathbb{Z})\). The layer exactly preserves the group manifold and Haar volume, preserves the commuting locus \(q_1q_2=q_2q_1\), and reproduces toral hyperbolic dynamics there, giving a structured long-horizon prior instead of an unconstrained matrix recurrence.

Useful6/10
Difficulty5/10
Novelty8/10
Paper: Quaternionic Extensions of Hyperbolic Toral Automorphisms arXiv:2608.08252
Unverified 2026

Wasserstein-Gated Cached Equilibrium Adapter

Add a small constrained equilibrium layer whose response depends on the current neural state and recent exogenous history, then cache responses keyed by a learned history embedding. For a new history, reuse a cached response only when an empirical Wasserstein distance to the cached history is below a threshold; otherwise run a few inner optimization iterations. The paper's local Holder and trajectory-stability results motivate graceful degradation rather than catastrophic errors for nearby…

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Stability of Differential Stochastic Variational Inequalities with History-Dependent Responses and Transfer Learning arXiv:2608.06923
Unverified 2026

Random-Batch Interaction Layer

Replace an all-pairs interaction or attention-like message field with an unbiased estimator obtained from a uniformly random partition into batches. Keep the partition fixed for \(\tau\) integration steps, then redraw it; the paper's error law predicts that too-long reuse causes a quadratic-in-\(\tau\) error accumulation, while finite-population error decreases as \(N^{-1}\).

Useful6/10
Difficulty4/10
Novelty5/10
Paper: Mean field error estimate of the random batch method for vortex blob dynamics for the 2D Navier--Stokes Equation arXiv:2608.06533
Unverified 2026

Invertible Fourier Surrogate for Periodic Sequence Modeling

Represent periodic input-output behavior using a compact real vector of Fourier coefficients and learn an invertible neural map from input coefficients to output coefficients. Inference then obtains the input representation for a desired periodic output by a single inverse pass instead of iterative optimization through a nonlinear forward model, while the Fourier representation reduces sequence dimensionality when high-rate signals are spectrally sparse.

Useful6/10
Difficulty6/10
Novelty4/10
Paper: Certified Feedforward Tracking for Unknown Nonlinear Systems via Invertible Neural Networks arXiv:2608.06419
Unverified 2026

Contour-reduced parametric SSM

Replace repeated full-dimensional matrix-exponential or ODE solves in a conditioned continuous-time state-space layer with contour quadrature evaluated in a projection basis. The same reduced basis and contour nodes can serve many conditioning vectors, while shifted reduced resolvents provide a stable and differentiable approximation over a prescribed time window.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Contour integral methods and model order reduction for parametric linear control systems arXiv:2608.05363
Unverified 2026

KAM-Stabilized Quasiperiodic Recurrent Memory

Construct a recurrent module with a phase variable and a transverse memory coordinate modeled on a perturbed twist map. Train the transverse state to lie on an invariant graph over the phase, while the phase follows an approximately irrational rigid rotation. A KAM-inspired graph correction and residual penalty should reduce long-horizon drift in recurrent prediction.

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Persistence of invariant graphs for twist maps under analytic perturbations arXiv:2608.05239
Unverified 2026

Chain-Current Latent States

Replace an unconstrained geometric latent vector with a state consisting of discrete chain coefficients, a continuous current, and an integral-current curvature. Neural updates are projected through the differential-homology boundary operator, so learned states remain compatible with conservation and boundary structure on meshes or point clouds.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Differential Homology arXiv:2608.05048
Unverified 2026

Automaton-Gated Objective Switching

Augment a neural policy with deterministic DFA states for the task objective and safety constraint, then select among objective-specific policy heads using those states. Before either target is reached, execute a mixed policy; after one target is reached, switch permanently to the policy specialized for the remaining target.

Useful6/10
Difficulty4/10
Novelty5/10
Paper: Optimal Constrained sc-LTL Planning in MDPs via Switching Policies arXiv:2608.05021
Unverified 2026

Random-start learned AMP

Construct an unrolled phase-retrieval network that begins with an isotropic Gaussian estimate rather than a spectral initializer. Retain the AMP residual correction and Onsager subtraction, but learn the scalar measurement denoisers and step sizes; use several random starts and select the iterate with the lowest measurement residual.

Useful6/10
Difficulty5/10
Novelty4/10
Paper: Approximate Message Passing with Random Initialization for Phase Retrieval arXiv:2608.01654
Unverified 2026

Contracting Countable-Branch Router

Construct a routed neural state update from a collection of branch maps whose inverse-style refinement operators are uniformly contractive. The discrete routing sequence acts as an expandable symbolic code, while contraction makes the continuous state associated with a long routing history insensitive to initialization and earlier perturbations. Use a finite active top-\(K\) set during training, but retain an expandable branch table so the model can represent increasingly complex or rare modes.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Infinite-Piecewise Expanding Maps: Chaos, Ergodicity and Invariant-Set Complexity arXiv:2608.00398
Unverified 2026

Invariant-Preserving Latent Compression

Replace unconstrained low-rank compression of a neural state with an augmented basis that always contains vectors representing known conserved quantities or diagnostically important linear statistics. After each learned transition, project the state back onto the affine constraint set with an exact minimum-norm correction, preventing rank truncation and model error from accumulating in those statistics.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Structure-Preserving Dynamical Low-Rank Approximations for Stochastic Vlasov--Poisson Equations arXiv:2608.00397
Unverified 2026

Null-form quadratic wave layer

Replace an unconstrained quadratic interaction between channel derivatives with a learnable combination of Lorentzian and antisymmetric null forms. For wave-equation surrogates, this enforces exact cancellation when two interacting features have parallel null directions, suppressing resonant derivative products that otherwise cause unstable long-horizon rollouts.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Recovery of a Null Form in the Wave Equation from Scattering Data arXiv:2607.28917
Unverified 2026

Normalized Scheduling-Degree Truncation

Use normalized scheduling variables and explicitly cap the degree of their products in a neural LPV or mixture-of-dynamics model. Instead of allowing every multiplicative interaction between scheduling coordinates and past or future features, retain only monomials below a chosen degree threshold. This produces a controllable approximation knob between a purely linear model and a full lifted predictor, while avoiding unstable extrapolation caused by poorly scaled high-degree features.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: A subspace approach to data-driven predictive control for linear parameter-varying systems arXiv:2607.28490
Unverified 2026

Port-Lifted Dynamics Network

Represent every predicted displacement and velocity as the sum of a prescribed boundary lift and a learned residual that is identically zero on the Dirichlet boundary. Feed the boundary velocity into the model through an explicit distributed-port feature and train an energy-balance residual so that the learned interior dynamics cannot inject arbitrary energy at the constrained boundary. This should eliminate boundary drift and reduce the burden on penalties or projection layers.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Strong imposition of Dirichlet boundary velocities in structure-preserving discretizations of elastodynamics arXiv:2607.26248
Unverified 2026

Low-Order Robust Functional Observer

Attach a small dynamical observer to a neural ODE, RNN, or state-space model and make it estimate only a task-relevant functional of the hidden state, such as logits, value features, or control-relevant projections. Use an incremental quadratic constraint and a bounded-real penalty to make the observer robust to hidden-state nonlinearities and input disturbances, instead of reconstructing the full latent state.

Useful6/10
Difficulty6/10
Novelty5/10
Paper: Functional H_infinity Filtering for Descriptor Systems with Incrementally Quadratic Nonlinearities under Disturbances arXiv:2607.25000