Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✗ Mechanism failed 2026

Pisot-Orbit Deterministic JL Layer

Replace a dense random projection used before retrieval, classification, or expert routing with a publicly reproducible matrix generated by a Pisot beta-transformation orbit. Search over a small public seed and sampling gap to select one matrix that preserves the calibration set's pairwise distances, then freeze it for training and inference. The projection removes random-matrix storage and makes the same embedding transform exactly reproducible across servers or proof systems.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Deterministic Johnson--Lindenstrauss Projections from Pisot $β$-Transformations for Zero-Knowledge Private Routing arXiv:2608.13078
✗ Mechanism failed 2026

Regret-aware evidential cost compression

Attach an evidential cost head to a neural graph model, representing each edge cost by a weighted set of interval boxes, and compress this representation before the downstream shortest-path or routing solver. Instead of minimizing Jaccard or Jousselme distance between the original and compressed mass functions, choose merges that minimize the induced cost error on the currently selected route, while enforcing a conservative monotonicity condition so that the resulting path regret is bounded.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: Decision-Aware Approximation of Belief Functions for Evidential Combinatorial Optimization arXiv:2608.10650
✗ Mechanism failed 2026

FFT Weak-Residual Engine

Evaluate weak residuals against a bank of periodic trigonometric test functions using FFT projections instead of repeated pointwise quadrature or output automatic differentiation. Frequency truncation and mode weighting provide a direct way to control the spatial scales enforced during neural PDE training.

Useful7/10
Difficulty4/10
Novelty6/10
Paper: Efficient Weak-Entropy PINN for Solving Hyperbolic Conservation Laws arXiv:2608.10389
✗ Mechanism failed 2026

Signed-Distance Ambient FNO

Embed every variable-domain sample into one fixed ambient hyperrectangle and append its signed-distance function as an additional channel to the operator input. Deterministically extend fields outside the physical domain, resample them onto a shared latent grid, apply standard Fourier layers, then interpolate and mask the output on the requested target discretization. The network learns the operator rather than a separate geometry encoder, so the same weights can be used across shapes and mesh…

Useful7/10
Difficulty4/10
Novelty6/10
Paper: ADEx-FNO: A Unified Ambient-Domain Framework for Fourier Neural Operators on Varying Geometries arXiv:2608.08608
✗ Mechanism failed 2026

RG Spectral Feature Gate

Replace fixed PCA-rank selection in a hidden layer with a renormalization-group-inspired gate over covariance eigenvalue bands. The gate retains modes whose effective quartic interaction remains unstable or strongly scale-dependent, while pruning bands that flow toward the Gaussian noise fixed point. Unlike top-eigenvalue truncation, this is designed for extensive-rank signal distributed throughout the bulk spectrum.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Data Field Theory: Theory and Applications of the Functional Renormalization Group for Signal Detection arXiv:2607.27236
✗ Mechanism failed 2026

Volume-Threshold Contracting State Layer

Construct a recurrent or state-space layer as a skew product: an expanding bounded feature coordinate drives a linearly contracting hidden state. Constrain the hidden transition matrix A to have spectral radius below one, and monitor the predicted transition ell times the absolute determinant of A equals one: below it, hidden trajectories should occupy a thin or fractal set, while above it they should have substantially higher-dimensional state coverage without losing local contraction.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Geometric Properties of Higher Dimensional Solenoidal Attractors arXiv:2607.27089
✗ Mechanism failed 2026

Shared Symbolic Mechanism Bottleneck

Replace the shared hidden trunk of a multi-output regression network with a small bank of differentiable symbolic units, then let every output use a sparse additive or multiplicative combination of the same units. The architecture explicitly tests whether outputs share a latent mechanism instead of merely sharing arbitrary neural features, improving identifiability and producing equations that can be inspected or exported.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Shared Symbolic Backbones for Physically Consistent Multi-Output Symbolic Regression arXiv:2607.26528
✗ Mechanism failed 2026

Autocatalytic Hysteresis Memory Cell

Replace or augment a recurrent hidden coordinate with a nonnegative bistable autocatalytic state driven by an external control signal. The cell retains information through metastable low and high states, while a periodic or slowly varying control produces a controlled phase lag and hysteresis useful for temporal regime detection. Explicit noise can be injected to test whether it enhances switching near the predicted intermediate-frequency regime.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Dynamic hysteresis in an autocatalytic reaction network arXiv:2607.24163
✗ Mechanism failed 2026

Dyadic Resolvent Filter Layer

Replace a single polynomial graph filter or dense inverse with a multiscale layer that applies unitary propagations at dyadic times and combines them according to the dyadic resolvent identity. For a symmetric graph operator, this implements a stable rational spectral filter that can selectively retain or suppress frequency bands while exposing logarithmic multiscale structure.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Dyadic Resolvent Representations of Self-Adjoint Operators: Propagator Expansions, Spectral Measures, and Zeta Functions arXiv:2607.21278
✗ Mechanism failed 2026

Unconstrained Proper-Velocity Hyperbolic Layers

Replace Lorentz-hyperboloid tensors with proper-velocity tensors whose spatial coordinates can be transformed by standard Euclidean affine layers and activations. Reconstruct the Lorentz time coordinate only at manifold boundaries, preserving the hyperbolic representation while avoiding repeated projection, normalization, or fragile exponential-map calculations.

Useful7/10
Difficulty4/10
Novelty6/10
Paper: Riemannian Deep Learning: Modules, Networks, and Geometries arXiv:2607.19305
✗ Mechanism failed 2026

Critical Spectral Mode Compression

Replace a large diagonalizable recurrent or state-space transition operator by a sparse set of retained oscillatory modes selected according to their contribution to the output autocorrelation. Unlike magnitude-based pruning, the objective is to preserve the power-law return signal generated by pairwise spectral differences, enabling long memory with far fewer modes.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Information Compression at Criticality arXiv:2607.18388
✗ Mechanism failed 2026

KS-Adaptive Graph Halting

Use the KS ratio to decide how many message-passing layers to execute per graph or per node, rather than selecting a fixed depth. In the subcritical regime, stop once the predicted remaining effect is below a tolerance; in the supercritical regime, continue until the observed logit change becomes small or a larger budget is reached.

Useful7/10
Difficulty5/10
Novelty5/10
Paper: The Value of Depth in Message Passing on Sparse Graphs: A Kesten-Stigum Dichotomy arXiv:2607.16676
✗ Mechanism failed 2026

Memory-Retaining RG Feature Blocks

Replace scale-blind pooling or downsampling with a coarse-graining block that carries an explicit relevant scale variable \(\eta\) alongside the feature field. The block is constrained to represent features in the memory-retaining form \(h(\xi,\eta)=\eta^{\alpha}F(\xi/\eta^{\beta})\), allowing both feature amplitude and profile shape to depend on the scale inherited from the input or previous RG step.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Memory Retention and the Classification of Renormalization-Group Fixed Points in Self-Similar Dynamics arXiv:2607.14388
✗ Mechanism failed 2026

Zeta-Corrected Singular Integral Layer

Construct a periodic neural integral layer whose fixed singular kernel behaves like |y|^{-s} near the origin, but whose samples on the uniform grid are replaced on a small symmetric stencil by SinCoTrap correction weights. The correction cancels low-order Taylor errors caused by sampling the singularity, while all nonlocal grid points remain unchanged. Increasing the correction order from p=0 to p=1 or p=2 should reduce discretization error without increasing global grid resolution.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: SinCoTrap: A High-Order Locally Corrected Trapezoidal Rule for Periodic Singular Integrals in Arbitrary Dimensions arXiv:2607.12390
✗ Mechanism failed 2026

Static Auxiliary-Graph Ensemble

Run the same neural decoder over several algebraically equivalent augmented graphs and aggregate their variable-level predictions. Each graph exposes different cycle structure and message routes, providing structured architectural diversity rather than ordinary random-seed ensembling.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Auxiliary Nodes for BP Decoding of Quantum LDPC Codes arXiv:2607.12187
✗ Mechanism failed 2026

Equivalent-Constraint Message Passing

Add auxiliary constraint nodes generated from linear combinations of existing constraints, creating a new message-passing graph while preserving the original feasible error set. Use a neural BP layer on the augmented graph so auxiliary nodes provide alternate paths around harmful cycles without changing the target constraints.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Auxiliary Nodes for BP Decoding of Quantum LDPC Codes arXiv:2607.12187
✗ Mechanism failed 2026

Critical power-law sparse attention

Replace dense self-attention by a sparse mask on a one-dimensional or ordered token geometry, retaining local neighbors and adding long-range edges with probability proportional to distance raised to \(-(1+\sigma)\). Use \(\sigma\approx0.8\text{--}0.85\) as the initial regime because the paper finds that this range supports delocalized, GOE-like connectivity despite sparse bonds. The resulting layer has linear or near-linear attention cost while maintaining long-range paths.

Useful7/10
Difficulty4/10
Novelty6/10
Paper: Emergent quantum chaos from correlations on a random graph arXiv:2607.11662
✗ Mechanism failed 2026

Volterra BBP multi-timescale recurrent initialization

Initialize a recurrent or state-space transition matrix with weak Wigner noise plus a shared cumulative-sum correlation structure. Increasing the correlation strength produces recurrent eigenmodes one at a time at analytically predicted BBP thresholds, yielding a controllable hierarchy of short- and long-memory modes. The matrix should then be globally rescaled or constrained so that all active modes remain inside the desired stability radius.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: A Cascade of Volterra-Operator BBP Transitions in a Correlated Wigner Matrix arXiv:2607.10503
✗ Mechanism failed 2026

Multi-view cycle-consistent matching layer

Replace independent pairwise feature matching across augmented or multimodal views with jointly estimated soft permutation matrices constrained to agree through cycles. The paper's multi-view result suggests that independent copies can cross a correspondence-recovery threshold even when every individual pairwise matching is statistically non-informative. In a neural network, this can provide cleaner token, patch, object, or cell alignment targets and can be used either as a differentiable…

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Geometric planted matchings in high dimensions: The power of multiple views arXiv:2607.09026
✗ Mechanism failed 2026

Monte Carlo Proximal Activation

Replace an expensive proximal activation or implicit optimization layer with a Gaussian barycentric estimator computed from energy evaluations. The resulting map is smooth and has a provable cocoercivity guarantee when the energy is weakly convex, making it a stable alternative to unconstrained learned activations or iterative proximal solvers.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Sharp bounds for stochastic proximal and projection estimators via radial dominance arXiv:2607.08670
✗ Mechanism failed 2026

Dual-Residual Depth Refinement

Train a residual network on a coarse depth mesh, estimate a dual-weighted residual for every layer interval, and insert new layers at intervals with the largest estimated contribution to objective error. This replaces uniform depth expansion or expensive neural architecture search with targeted refinement driven by both forward-dynamics error and downstream loss sensitivity.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: An optimal control approach for neural network architecture adaptation with a posteriori error estimation arXiv:2607.07637
✗ Mechanism failed 2026

DMDc-Initialized Marginal-Stable Neural SSM

Replace the unconstrained transition of a recurrent or state-space neural network with a DMDc-initialized linear latent transition plus a learned nonlinear residual. Estimate the transition from a short warm-up dataset using Hankel delay coordinates, retain eigenmodes with decay rates near the unit circle for long-term memory, and let the neural residual model dynamics not explained by the linear backbone. This should make long-horizon prediction and slowly varying signals easier to learn while…

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Koopman Spectral Analysis of Lithium-Ion Battery Dynamics: State of Charge as a Marginally Stable Observable arXiv:2607.07594
✗ Mechanism failed 2026

Exact Multi-Population Hyperplane Router

Replace a learned binary MoE gate with a hyperplane whose two sides contain prescribed proportions of several token populations simultaneously. In a low-dimensional routing projection, solve the cap-volume equations directly, producing deterministic per-population load control without an auxiliary load-balancing loss. Recursively applying the construction yields a balanced binary expert tree.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: From Ham-Sandwich to Centerpoints: Semialgebraic Algorithms for Cutting Polytopal Measures arXiv:2607.02400
✗ Mechanism failed 2026

Nonlinear Laplacian Equilibrium GNN Layer

Replace several fixed message-passing layers with an implicit graph layer whose state is the solution of a nonlinear flow equilibrium. Learn monotone edge laws from endpoint features, solve for node potentials with damped chord-Newton steps, and use the resulting edge flows or potentials as the layer output. Monotonicity and the Laplacian Jacobian provide a principled stability mechanism while retaining sparse graph computation.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: NLF: A Resistor-Network Framework and Linear-Time Solver for Convex Network-Flow Equilibria arXiv:2607.02041