Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✓✓ Beats tuned baseline 2026

Inverse-Gain Structured Privileged Distillation

Replace direct action imitation with a causal recurrent estimator of the inverse input gain. The neural network predicts the latent quantity needed by the expert controller, and a fixed algebraic wrapper converts that prediction into an action using the measured state difference and tracking error, thereby removing the additive disturbance exactly under the sampled timing model.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: From Privileged Control to Deployable Adaptation:Fusing Mechanism-Guided Task Reduction with Learned Behavior arXiv:2608.10453
△ Mechanism confirmed, baseline not beaten 2026

Recursive Active-Frontier Policy Shield

Replace a binary action mask with a lossless probability-preserving shield for sequential neural decisions. At state s, retain only actions that preserve current and future feasibility, then transfer the raw policy mass of excluded actions to admissible frontier actions according to process distance and the actor's existing safe-action preference.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: Process-Knowledge-Embedded Safe DRL for Real-Time Dispatch of Process Loads in Industrial Microgrids arXiv:2608.03149
✗ Mechanism failed 2026

Orbit-Consistent Equivariant Distillation

Constrain a student policy to transform its action in the same way that the input state is transformed, while constraining its value estimate to remain unchanged. During distillation, augment every teacher-student pair with several symmetry-transformed copies and penalize disagreement after transforming the student action back to the original frame.

Useful7/10
Difficulty5/10
Novelty4/10
Paper: SymVD: Symmetric Vision Language Action Distillation for Robot Manipulation arXiv:2608.29828
✗ Mechanism failed 2026

Variable-Projection Adaptive Fourier Layer

Replace fixed random Fourier features in a coordinate network or PINN with a small complex Fourier dictionary whose propagation directions are learned from a weighted residual. Given directions, solve the linear feature coefficients exactly or by ridge regression, and optimize only the directions in the outer loop. This should represent low-directional-complexity fields with fewer features and avoid wasting gradient updates on coefficients that can be fitted analytically.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Direction-Adaptive Plane-Wave Discontinuous Galerkin Methods for the Helmholtz Equation arXiv:2608.18380
✗ Mechanism failed 2026

Automaton-Supervised Neural Policy Shield

Attach a deterministic supervisory automaton to a neural policy or sequence model and mask every event disabled by the current supervisor state. Use a short receding-horizon planner over admissible events to resolve conflicts between neural preferences and shared-resource constraints. The network scores useful actions, while the automaton supplies an exact safety layer.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Model Predictive Supervisory Control for Hierarchical and Distributed UAS Traffic Management arXiv:2608.18353
△ Mechanism confirmed, baseline not beaten 2026

Persistent Workspace for Online Adaptation

Turn the latent substrate into a persistent computational workspace for sequential inputs: each new observation is written into a designated subspace, processed by the same local rule, decoded, and then selectively retained or reset. This creates a compact recurrent model whose state can accumulate algorithmic information across a stream without expanding the parameter count.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Emergent Models: Intelligence from Tiny Substrates arXiv:2608.14019
△ Mechanism confirmed, baseline not beaten 2026

Intrinsic-Rank Filter Memory for Actor-Critic

Replace an oversized recurrent hidden state or raw history stack with a causal filtered input-output lift followed by an SVD-selected bottleneck. The actor, critic, and Bellman regression operate only on the identifiable memory coordinates, preventing deterministic null directions from being fitted as if they were independent state variables.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Reinforcement Learning-Based Output Feedback LQR for Continuous-Time MIMO Systems arXiv:2608.11750
△ Mechanism confirmed, baseline not beaten 2026

Tropical Multisymmetric Pooling

Replace ordinary sum or mean pooling in a permutation-invariant set network by the complete family of basic tropical multisymmetric values. For an input set of n points in R^r, each feature computes the maximum total coordinate score obtainable by assigning disjoint rows to prescribed coordinate channels. The resulting representation is invariant to row permutations, separates all multisets, and inherits a bi-Lipschitz relation to optimal row matching, so nearby sets cannot be arbitrarily…

Useful7/10
Difficulty5/10
Novelty8/10
Paper: The basic tropical polynomials generate the semifield of $r$-symmetric tropical rational functions arXiv:2608.06857
✗ Failed on benchmark 2026

Exact Event-Chained Neural ODE

Represent a hybrid trajectory with one neural module per known dynamical phase rather than a single network spanning all phases. Feed the predicted terminal state of phase r directly as the initial state of phase r+1, so continuity is satisfied by construction instead of by a soft interface penalty. This should improve learning near abrupt changes and remove an otherwise poorly conditioned loss-weight tradeoff.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Event-Structured Physics-Informed Neural Networks for Differentiable Critical Clearing Boundaries arXiv:2607.27681
✗ Mechanism failed 2026

Shared Symbolic Mechanism Bottleneck

Replace the shared hidden trunk of a multi-output regression network with a small bank of differentiable symbolic units, then let every output use a sparse additive or multiplicative combination of the same units. The architecture explicitly tests whether outputs share a latent mechanism instead of merely sharing arbitrary neural features, improving identifiability and producing equations that can be inspected or exported.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Shared Symbolic Backbones for Physically Consistent Multi-Output Symbolic Regression arXiv:2607.26528
△ Mechanism confirmed, baseline not beaten 2026

Feasible Action Mapping Safety Layer

Let a neural policy emit an unconstrained abstract action z, then solve a state-dependent feasibility problem that maps z to an admissible optimal-control parameter p before execution. Unlike coordinate-wise clipping, the mapping accounts for predicted dynamics, coupled state and input constraints, and recursive feasibility, allowing the policy to retain a simple unconstrained output space while the controller enforces plant constraints.

Useful7/10
Difficulty6/10
Novelty5/10
Paper: Bridging Reinforcement Learning and Optimal Control via Feasible Action Mapping arXiv:2607.23930
△ Mechanism confirmed, baseline not beaten 2026

Tangent-Branch Neural Evasion Layer

Wrap a neural multi-agent policy with an analytic planner that generates turn-straight trajectories tangent to pursuer surveillance disks, then selects the branch with the smallest predicted completion time. The network supplies high-level preferences or residual corrections, while the geometric layer prevents unnecessarily entering exclusion regions and exposes an explicit branch-switching signal for training.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Semi-Explicit Solutions to the Prying-Pedestrian Surveillance-Evasion Differential Game and Extensions to Two Pursuers arXiv:2607.21087
△ Mechanism confirmed, baseline not beaten 2026

Rank-One Delta Associative Memory

Replace a portion of quadratic key-value attention or an external episodic table with a per-sample matrix fast memory updated by rank-one delta corrections. The memory directly learns a linear key-to-value map and can be carried across sequence segments, providing cheap online adaptation with constant state size per head.

Useful7/10
Difficulty5/10
Novelty5/10
Paper: Memoir: Should a Model Write to Its Memory While It Thinks? arXiv:2607.20792
✓✓ Beats tuned baseline 2026

Geometric Feedback Compute Scheduler

Treat unresolved inference items as active threats and allocate a fixed budget of C module evaluations per round. Each evaluation has an item-dependent probability of completing the item, while the scheduler observes only completion or failure after the round. Use fair allocation when completion probabilities are unknown or nearly homogeneous, then switch to a marginal-success greedy policy as feedback estimates become reliable.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Meeting Uncertain Threats with Feedback arXiv:2607.13648
△ Mechanism confirmed, baseline not beaten 2026

Monotone Forced-Response Adapter

Separate a pretrained sequence model's passive prediction from the causal effect of an action, and learn only the latter with a compact monotone adapter. The adapter receives the current latent state and an action deviation, but its action-to-output Jacobian is constrained to have the physically correct sign, preventing intervention predictions that move opposite to the applied control.

Useful7/10
Difficulty4/10
Novelty7/10
Paper: ThermoForce: A Physics-Structured Interventional World Model for Building HVAC Control arXiv:2607.03942
✗ Mechanism failed 2026

Minimum-eigenvalue spectral pruning

Replace magnitude-based channel or expert pruning with a subset-selection objective that maximizes the weakest direction in the candidates' activation span. Relax the binary mask to continuous gates, optimize an entropic soft minimum eigenvalue, and round the gates to retain a fixed number of channels or experts. This should preserve diverse representations and reduce redundant feature directions.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Discrete eigenvalue optimization from entropic smoothing and first-order methods arXiv:2608.27024
Audited (legacy) 2026

Spectral Budgeted Embeddings

Replace uniform embedding dimensions with a globally budgeted allocation based on the estimated spectral complexity of each categorical feature. Tables whose category representations have large leading singular-value energy receive more dimensions, while high-cardinality tables are penalized because each extra dimension consumes more parameters.

Useful7/10
Difficulty4/10
Novelty6/10
Paper: Optimal Allocation of Embedding Dimensions under Finite-Sample Constraints arXiv:2608.24592
Unverified 2026

Bernstein-Safe Policy Head

Replace a neural controller's pointwise action outputs over a finite horizon with Bernstein control points whose convex hull satisfies actuator and trajectory constraints. The network predicts the control points, while a robust margin accounts for bounded tracking or model-prediction error, making continuous-time actuator feasibility checkable from finitely many inequalities.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Actuator-Aware Spatiotemporal Tube Synthesis for Temporal Reach-Avoid-Stay Tasks arXiv:2607.23040
Unverified 2026

Mass-Conserving Puncta Router

Insert a differentiable reaction-diffusion layer that converts dense token or pixel features into sparse, spatially coherent routing masks. Two competing orientations form complexes through conserved monomer reservoirs, so local assignments can cluster while opposite assignments mutually exclude one another instead of independently activating at the same location. The layer can be used as a soft-to-hard MoE router, attention-mask generator, or object-part grouping module.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Clustering versus sorting: a mass-conserving reaction-diffusion model of planar polarity puncta arXiv:2608.29679
Unverified 2026

Leaf-Relocation Tree Router

Learn a branching hierarchy for tokens, examples, or experts by greedily relocating leaves to reduce T-Robinson violations. The resulting tree supplies hierarchical candidate sets for retrieval or MoE routing, allowing the model to search a small subtree instead of all items while adapting the hierarchy to learned representations.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: T-Robinson Spaces: Structure, Recognition, and Applications to Real Data arXiv:2608.21248
Unverified 2026

Scalar Residual Thermodynamic Head

Attach a small temperature-pressure residual head to a pretrained structural encoder instead of relearning the full free-energy surface. Predict one scalar Gibbs free energy and obtain entropy, volume, and other thermodynamic responses by automatic differentiation, enforcing that all outputs derive from a common potential.

Useful6/10
Difficulty4/10
Novelty6/10
Paper: Universal Thermodynamic Interatomic Potentials for Crystalline Materials arXiv:2608.14502
Unverified 2026

Free-boundary neural halting

Replace a fixed confidence-threshold early-exit rule with a finite-horizon optimal-stopping policy over the model's evolving posterior confidence. The controller stops when the calibrated expected terminal error is no greater than the cost plus expected value of executing another neural block, permitting time-dependent and nonmonotone stopping regions.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: When should one stop the most exciting game? Sequential Inference for win-martingales arXiv:2608.12291
Unverified 2026

Ambiguity-Flat Weyl Feature Layer

Replace a random cyclic filter bank or patch projection with the Weyl–Heisenberg orbit of one normalized learnable prototype. Regularize the prototype so that all nonzero shift and modulation correlations have a large and nearly equal magnitude, maximizing the smallest eigenvalue of the induced feature Gram matrix and preventing poorly observed feature directions.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Uniformly Stable Minimal Weyl--Heisenberg Measurements Approaching the SIC Benchmark arXiv:2608.11850
Unverified 2026

Prototype Distance-Field Safety Layer

Store a finite library of successful robot configurations or action-conditioned waypoints and construct a smooth soft minimum of their distances. Use the negative distance gradient as a structured action prior, add a learned residual policy, and pass the combined action through a quadratic-program safety layer. This gives a neural controller an explicit attraction basin toward demonstrated solutions while preventing violations of known state constraints.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Grasp Execution Without a Planner: Configuration-Space Grasp Distance Fields with Certified Safety & Guaranteed Quality arXiv:2608.00600