Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

△ Mechanism confirmed, baseline not beaten 2026

Critical-Region LP Policy Layer

Replace black-box differentiation through an embedded LP decision with an analytic Jacobian computed from the LP’s active basis. A neural policy emits LP coefficients or right-hand sides; the LP returns the decision, while the backward pass uses the basis inverse and dual sensitivity, avoiding solver unrolling and finite-difference noise.

Useful8/10
Difficulty5/10
Novelty5/10
Paper: Simulation-Optimization of Systems of Optimizers: Exploiting the Inner Optimization's Geometry arXiv:2608.18129
△ Mechanism confirmed, baseline not beaten 2026

Positive Curvature Mirror-Descent DEQ

Replace the black-box equilibrium denoiser in an image-restoration DEQ with a positivity-preserving mirror-descent equilibrium driven by the exact Gamma likelihood and a discretized surface-area/mean-curvature regularizer. The equilibrium layer has a small number of learned scalar or channel-wise parameters instead of a large implicit CNN, while the exponentiated update prevents negative intensities and naturally matches multiplicative noise.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: A geometry-based deep equilibrium model for image restoration under multiplicative Gamma noise arXiv:2608.04944
△ Mechanism confirmed, baseline not beaten 2026

Recursive Active-Frontier Policy Shield

Replace a binary action mask with a lossless probability-preserving shield for sequential neural decisions. At state s, retain only actions that preserve current and future feasibility, then transfer the raw policy mass of excluded actions to admissible frontier actions according to process distance and the actor's existing safe-action preference.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: Process-Knowledge-Embedded Safe DRL for Real-Time Dispatch of Process Loads in Industrial Microgrids arXiv:2608.03149
△ Mechanism confirmed, baseline not beaten 2026

Certified contraction implicit layer

Replace a deep feed-forward block by the fixed point z=phi(Wz+Vx+b), with the recurrent weight W constrained so that the fixed point is unique for every input. The same condition makes forward fixed-point iteration stable and makes implicit differentiation well-conditioned, allowing depth-independent memory usage while providing a measurable spectral failure boundary.

Useful8/10
Difficulty5/10
Novelty4/10
Paper: Implicit Neural Networks as Static Controllers: Certificates and Performance Separation arXiv:2607.11122
✓✓ Beats tuned baseline 2026

Spectral pinning of neural modules

Represent communicating layers, experts, or distributed workers as nodes of a weighted graph and apply strong corrective updates only to a small pinned subset. Select pins by the increase they produce in the grounded Laplacian smallest eigenvalue, because this spectral gap predicts the decay rate of representation disagreement.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Event-Triggered Pinning Impulsive Control of Complex Networks with Actuation Delays: Stability Analysis and Zeno-Free Conditions arXiv:2608.24074
△ Mechanism confirmed, baseline not beaten 2026

Quadrature-Whitened Neural Feature Subspace

Freeze a wide neural spatial dictionary, then compress and whiten it using the quadrature mass matrix before solving for output coefficients or latent PDE states. The retained basis removes feature directions that are numerically invisible or nearly dependent under the actual domain discretization, while preserving the represented function space up to the chosen SVD rank.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Evo-GTransNet for Parabolic PDEs: A Fixed-Feature Galerkin Method of Lines with Quadrature-Mass Orthonormalization arXiv:2608.19615
△ Mechanism confirmed, baseline not beaten 2026

Congestion-aware equimarginal MoE router

Replace independent token-to-expert softmax routing with a fixed-budget congestion game. Each token group distributes a fixed routing mass across experts, while the marginal value of an expert decreases as other groups send mass there. Iteratively route toward the highest current marginal utility and exploit sorted-prefix supports to produce sparse, capacity-aware assignments.

Useful7/10
Difficulty5/10
Novelty5/10
Paper: The Game of Marginal Utilities arXiv:2608.10373
✗ Mechanism failed 2026

Volume-Threshold Contracting State Layer

Construct a recurrent or state-space layer as a skew product: an expanding bounded feature coordinate drives a linearly contracting hidden state. Constrain the hidden transition matrix A to have spectral radius below one, and monitor the predicted transition ell times the absolute determinant of A equals one: below it, hidden trajectories should occupy a thin or fractal set, while above it they should have substantially higher-dimensional state coverage without losing local contraction.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Geometric Properties of Higher Dimensional Solenoidal Attractors arXiv:2607.27089
△ Mechanism confirmed, baseline not beaten 2026

Feasible Action Mapping Safety Layer

Let a neural policy emit an unconstrained abstract action z, then solve a state-dependent feasibility problem that maps z to an admissible optimal-control parameter p before execution. Unlike coordinate-wise clipping, the mapping accounts for predicted dynamics, coupled state and input constraints, and recursive feasibility, allowing the policy to retain a simple unconstrained output space while the controller enforces plant constraints.

Useful7/10
Difficulty6/10
Novelty5/10
Paper: Bridging Reinforcement Learning and Optimal Control via Feasible Action Mapping arXiv:2607.23930
✗ Failed on benchmark 2026

Volume-Mass Diffusion GNN

Replace ordinary graph propagation by diffusion with a positive node-dependent mass matrix \(\mathbf V\), so high-volume nodes update slowly and low-volume nodes update rapidly. Use node volumes as fixed metadata, a function of degree, or learned positive gates; this makes the architecture sensitive to dynamical localization that degree-normalized GCNs cannot represent.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Localization transitions of diffusion dynamics in physical networks arXiv:2607.19486
△ Mechanism confirmed, baseline not beaten 2026

Gain-Weighted Cluster Co-Design

Use small-gain diagnostics to jointly learn module normalization and a communication partition rather than imposing a fixed global spectral constraint. Clusters should be formed around high-gain feedback loops, because grouping weakly related modules cannot improve the certificate and only adds bookkeeping.

Useful7/10
Difficulty7/10
Novelty7/10
Paper: Cluster-Based Distributed Small-Signal Stability Certificates for Grid-Forming Inverter Networks arXiv:2607.16985
✗ Failed on benchmark 2026

ISS-Constrained Modular Recurrent Network

Replace an unconstrained recurrent block with two coupled modules: a contractive perceptual estimator and an input-to-state-stable cognitive state transition. Spectral normalization and a controlled Euler residual step enforce a quantitative gain condition, preventing hidden-state explosion while retaining long memory when the contraction factor is chosen close to one.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: A modular state-space model of human perception, cognition, and decision dynamics arXiv:2607.14078
△ Mechanism confirmed, baseline not beaten 2026

Mittag-Leffler Memory Diagnostic and Gating

Measure how validation forecast error grows with prediction horizon and fit exponential and Mittag-Leffler models. When the Mittag-Leffler fit is decisively better, activate a fractional-memory SSM or long-memory residual branch and use its fitted effective order to set the branch's kernel decay and horizon-loss weights; otherwise retain a conventional recurrent or finite-memory branch.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Mittag-Leffler-Type Forecast-Error Growth as a Diagnostic Indicator of Fractional Dynamics arXiv:2607.08588
Unverified 2026

Pressure-Controlled Neural IFS

Construct a generative or recurrent neural architecture with several contractive or mildly expanding branches, and explicitly control the geometric complexity of its invariant set using the sub-additive singular-value pressure of branch-Jacobian products. Instead of regularizing only the operator norm, the model can preserve anisotropic directions while targeting a desired attractor dimension, potentially improving coverage of structured data without uncontrolled folding or collapse.

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Dimensions of surface repellers and attractors of non-linear planar IFSs arXiv:2608.30744
Unverified 2026

Water-Filled Block-Sparse Neural Connectivity

Partition neural modules into two empirically identified reliability or noise classes and restrict their communication graph to a two-block stochastic block model. Allocate a fixed connectivity budget across within-class and cross-class edges using a water-filling update that favors block pairs producing the largest increase in validation utility. The resulting layer is sparse and modular, with a testable prediction that optimal connectivity concentrates on a few block pairs rather than…

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Graphon Design for Human-Machine Coordination under Bounded Rationality: Optimality of Stochastic Block Models arXiv:2608.27851
Unverified 2026

Incidence-Mixed Simplicial Diffusion

Represent edge or pair-token features and propagate them with a convex mixture of two normalized channels: transitions through shared vertices and transitions through shared triangles. This preserves higher-order connectivity that an ordinary graph convolution loses, while the mixing coefficient q controls whether information follows pairwise support or genuine triangular structure.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Incidence-based random walks on simplicial complexes arXiv:2608.17229
Unverified 2026

Åberg Percolation Routing

Replace a dense neural interaction graph by a dynamically activated graph whose edge $(u,v)$ is retained only when its effective coupling exceeds the local spacing of response modes. The network remains sparse below the connectivity transition but becomes globally communicating once a giant component forms, providing a controllable alternative to arbitrary magnitude pruning.

Useful6/10
Difficulty6/10
Novelty8/10
Paper: Small-world structure of quantum computer hardware arXiv:2608.13855
Unverified 2026

Random-start learned AMP

Construct an unrolled phase-retrieval network that begins with an isotropic Gaussian estimate rather than a spectral initializer. Retain the AMP residual correction and Onsager subtraction, but learn the scalar measurement denoisers and step sizes; use several random starts and select the iterate with the lowest measurement residual.

Useful6/10
Difficulty5/10
Novelty4/10
Paper: Approximate Message Passing with Random Initialization for Phase Retrieval arXiv:2608.01654
Unverified 2026

Prototype Distance-Field Safety Layer

Store a finite library of successful robot configurations or action-conditioned waypoints and construct a smooth soft minimum of their distances. Use the negative distance gradient as a structured action prior, add a learned residual policy, and pass the combined action through a quadratic-program safety layer. This gives a neural controller an explicit attraction basin toward demonstrated solutions while preventing violations of known state constraints.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Grasp Execution Without a Planner: Configuration-Space Grasp Distance Fields with Certified Safety & Guaranteed Quality arXiv:2608.00600
Unverified 2026

Structure-Preserving Profile Layer

Replace unconstrained output coordinates with a neural parameterization whose outputs are valid monotone profiles by construction, analogous to representing a Young diagram through nonnegative ordered row increments. Train the network against an explicit energy or negative log-probability while preserving the feasible geometry, rather than relying on penalties that permit invalid intermediate states.

Useful6/10
Difficulty4/10
Novelty6/10
Paper: Neural variational framework for random Young-diagram limit shapes arXiv:2607.27061
Unverified 2026

Double-Bracket Projector Refinement

Represent an attention or routing state as a symmetric projector or fixed-spectrum positive semidefinite matrix and refine it using the paper's double-bracket flow instead of unconstrained gradient steps. The update rotates the state toward a task-derived Hermitian cost matrix while preserving its eigenvalues, so rank, trace, and spectral diversity remain fixed by construction.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: A Path Integral Model of Cognition arXiv:2607.24807
Unverified 2026

Connectivity-aware ATC gradient tracking

Use the paper's mode decomposition to choose between ordinary DIGing and augmented ATC-DIGing/AugDGM according to the communication spectrum and curvature. The augmented scheme changes the disagreement feedback gain mode by mode, which can reduce the dominant pole radius on well-connected graphs without changing the neural-network architecture.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Exact Worst-case Convergence Rates of Distributed Gradient Tracking Methods arXiv:2607.23601
Unverified 2026

Hessian-guided orthogonal sparsification

Train a square orthogonal neural mixer while maximizing its entrywise fourth-power concentration. When optimization reaches a non-permutation stationary configuration, explicitly test rank-two row or column rotations and take a rotation with positive exact second variation, using the paper's constructive saddle-escape mechanism.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Local Maxima of the Entrywise $\ell_4$ Norm on the Orthogonal Group arXiv:2607.12431
Unverified 2026

Monotone Resolvent Elimination Layer

Build an implicit layer from a piecewise-linear maximal monotone operator on visible variables z_* and auxiliary variables z_**, then eliminate the auxiliary block rather than exposing it in the network output. Compute the layer through a fixed point of the eliminated component of a nonexpansive resolvent, with damping when the auxiliary map is not strictly contractive.

Useful5/10
Difficulty6/10
Novelty6/10
Paper: Maximal monotonicity of piecewise polyhedral mappings arXiv:2607.07358