Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✗ Mechanism failed 2026

Reference-Preserving Martingale Layer

Replace an unconstrained stochastic transition between categorical or discretized latent distributions by a transition matrix that preserves a prescribed reference distribution while mapping relative populations through a martingale. This prevents the layer from inventing arbitrarily sharp deviations from the reference and imposes a convex-order monotonicity condition on uncertainty across layers or diffusion time steps.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: State convertibility and fluctuation theorems from a dynamical reference: majorization meets martingales arXiv:2608.19391
✗ Mechanism failed 2026

Front-Calibrated Bistable Neural Field

Construct a spatial recurrent network whose local vector hidden state has two stable attractors and whose neighbor coupling is diffusive. Train or constrain the network so that the desired attractor invades the undesired one with a controlled positive front velocity, rather than relying on a scalar class-frequency variable that can erase depletion and interface structure.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Nucleation beyond Equilibrium: Fronts Control Invasion in Bistable Ecosystems arXiv:2608.05251
✗ Mechanism failed 2026

Permutation-family residual network

Replace direct learning of a highly cancelling signed observable with a quotient-space model over symmetry orbits of inputs. Predict a physically constrained baseline for each family and use an LSTM or set/graph encoder only for the residual many-body correlation, then aggregate family predictions with known signed weights instead of forming a noisy sample-level ratio.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Learning the Fermion sign structure in path-integral Monte Carlo arXiv:2607.15060
✗ Mechanism failed 2026

Correlated Long-Range Residual Mixer

Replace an iid local or randomly sparse residual mixer with a distance-decaying long-range operator whose edge magnitudes are correlated through a shared latent Gaussian field. The paper predicts that these correlations qualitatively change low-energy spectral scaling and increase multiscale information propagation relative to iid long-range weights. Apply the operator as a spectrally normalized residual block so that the benefit comes from correlated scale coverage rather than uncontrolled…

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Effective Hamiltonian description on monitored Majorana chains: correlated power-law hoppings and unconventional entanglement scaling arXiv:2609.04091
✗ Mechanism failed 2026

Orbit-Consistent Equivariant Distillation

Constrain a student policy to transform its action in the same way that the input state is transformed, while constraining its value estimate to remain unchanged. During distillation, augment every teacher-student pair with several symmetry-transformed copies and penalize disagreement after transforming the student action back to the original frame.

Useful7/10
Difficulty5/10
Novelty4/10
Paper: SymVD: Symmetric Vision Language Action Distillation for Robot Manipulation arXiv:2608.29828
✗ Mechanism failed 2026

Automaton-Supervised Neural Policy Shield

Attach a deterministic supervisory automaton to a neural policy or sequence model and mask every event disabled by the current supervisor state. Use a short receding-horizon planner over admissible events to resolve conflicts between neural preferences and shared-resource constraints. The network scores useful actions, while the automaton supplies an exact safety layer.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Model Predictive Supervisory Control for Hierarchical and Distributed UAS Traffic Management arXiv:2608.18353
✗ Mechanism failed 2026

Gaussian-mixture kinetic neural solver

Make a neural network predict a positive Gaussian-mixture representation of the distribution function rather than independent values on a momentum grid. Use the mixture parameters inside a differentiable Boltzmann collision operator, so training directly enforces the interaction mechanism and exposes the relaxation spectrum responsible for ballistic-to-hydrodynamic crossover.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Linear response across interaction regimes in two-dimensional ferromagnets arXiv:2608.14477
✗ Mechanism failed 2026

Volume-Threshold Contracting State Layer

Construct a recurrent or state-space layer as a skew product: an expanding bounded feature coordinate drives a linearly contracting hidden state. Constrain the hidden transition matrix A to have spectral radius below one, and monitor the predicted transition ell times the absolute determinant of A equals one: below it, hidden trajectories should occupy a thin or fractal set, while above it they should have substantially higher-dimensional state coverage without losing local contraction.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Geometric Properties of Higher Dimensional Solenoidal Attractors arXiv:2607.27089
✗ Mechanism failed 2026

Unconstrained Proper-Velocity Hyperbolic Layers

Replace Lorentz-hyperboloid tensors with proper-velocity tensors whose spatial coordinates can be transformed by standard Euclidean affine layers and activations. Reconstruct the Lorentz time coordinate only at manifold boundaries, preserving the hyperbolic representation while avoiding repeated projection, normalization, or fragile exponential-map calculations.

Useful7/10
Difficulty4/10
Novelty6/10
Paper: Riemannian Deep Learning: Modules, Networks, and Geometries arXiv:2607.19305
✗ Mechanism failed 2026

KS-Adaptive Graph Halting

Use the KS ratio to decide how many message-passing layers to execute per graph or per node, rather than selecting a fixed depth. In the subcritical regime, stop once the predicted remaining effect is below a tolerance; in the supercritical regime, continue until the observed logit change becomes small or a larger budget is reached.

Useful7/10
Difficulty5/10
Novelty5/10
Paper: The Value of Depth in Message Passing on Sparse Graphs: A Kesten-Stigum Dichotomy arXiv:2607.16676
✗ Mechanism failed 2026

Memory-Retaining RG Feature Blocks

Replace scale-blind pooling or downsampling with a coarse-graining block that carries an explicit relevant scale variable \(\eta\) alongside the feature field. The block is constrained to represent features in the memory-retaining form \(h(\xi,\eta)=\eta^{\alpha}F(\xi/\eta^{\beta})\), allowing both feature amplitude and profile shape to depend on the scale inherited from the input or previous RG step.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Memory Retention and the Classification of Renormalization-Group Fixed Points in Self-Similar Dynamics arXiv:2607.14388
✗ Mechanism failed 2026

Dual-Residual Depth Refinement

Train a residual network on a coarse depth mesh, estimate a dual-weighted residual for every layer interval, and insert new layers at intervals with the largest estimated contribution to objective error. This replaces uniform depth expansion or expensive neural architecture search with targeted refinement driven by both forward-dynamics error and downstream loss sensitivity.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: An optimal control approach for neural network architecture adaptation with a posteriori error estimation arXiv:2607.07637
✗ Mechanism failed 2026

Canonical Tropical Segment State

Build a sequence or graph module whose state is a canonical set of affine tropical pieces rather than an opaque hidden vector. Compose consecutive segments by Minkowski addition of their lifted supports, merge alternatives by union followed by lower-hull reduction, and evaluate the resulting piecewise-linear function with a minimum.

Useful7/10
Difficulty7/10
Novelty8/10
Paper: On the Representational Geometry of Dynamic Programs arXiv:2608.25034
✗ Mechanism failed 2026

Resolvent Fractional-Power Layer

Parameterize a learned feature-space operator as accretive but not necessarily symmetric, then apply its fractional power through a finite positive mixture of shifted resolvents. This provides a matrix-function layer that can represent directional and rotational interactions while avoiding unstable eigendecomposition of nonnormal matrices.

Useful6/10
Difficulty6/10
Novelty7/10
Paper: Functions and Means of Accretive Operators arXiv:2607.09152
✗ Mechanism failed 2026

Innovation-Compensated Latent Policy

In a partially observed reinforcement-learning or model-based control agent, expose the state-estimator innovation to the action head through a dedicated residual feedback branch. The policy produces a nominal action from the estimated latent state, while a learned innovation-compensation branch corrects actions when observations disagree with predicted latent dynamics. This explicitly separates nominal policy behavior from estimation-induced corrections and should help during fast transients…

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Revisiting Certainty Equivalence: The Structural Coupling Between Estimation and Control in Underactuated Nonlinear Systems arXiv:2607.07276
✗ Mechanism failed 2026

Diffeomorphic gauge-fixing layer

Insert a differentiable spatial canonicalization module before a neural dynamics model. It estimates a smooth invertible coordinate transformation that places each input field in a common gauge relative to a reference template, predicts the next state in that gauge, and maps predictions back to the original coordinates. The module should reduce the need for the dynamics network to relearn identical laws under many smooth spatial reparameterizations.

Useful6/10
Difficulty6/10
Novelty6/10
Paper: The Right Space for Dynamics: Numerics with Diffeomorphism Equivariance arXiv:2607.06536
✗ Mechanism failed 2026

Periodic CMV Unitary Recurrent Layer

Replace a dense recurrent transition matrix with a periodic CMV-style product of alternating local 2x2 unitary cores. The transition is exactly norm-preserving, has O(n) trainable parameters under periodic tying, and can be applied through local factor operations rather than stored as an n-by-n matrix. Use turnover refactorization when changing the ordering or boundary connection of cores, enabling a compact cyclic unitary state-space layer.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Fast computation of eigenvalues of periodic CMV matrices arXiv:2607.06400
✗ Mechanism failed 2026

Dual-unitary recurrent state block

Replace a generic recurrent transition with two coupled unitary transitions that share one block column and differ by a sign on the other block column. Each transition preserves hidden-state norm exactly, while the structured difference gives a controlled two-path recurrent architecture for long-context modeling.

Useful6/10
Difficulty5/10
Novelty5/10
Paper: Operator model and a trace formula for pairs of unitary operators arXiv:2607.05334
✗ Mechanism failed 2026

Pole-Certified SSM Initialization

Extract a small set of stable exponential modes from an observed neural sequence and use them to initialize a diagonal or block-diagonal state-space model. Hankel-pencil eigenvalues propose the modes, while persistence across shifts and contour margins reject modes caused by noise or a short-lived background.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Determinant Characteristics and Argument-Principle Certification for Visible Poles in Meromorphic Continuation arXiv:2607.04568