Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✓✓ Beats tuned baseline 2026

Path-Holonomy Attention

Replace ordinary additive path aggregation in graph attention with ordered products of edge operators equipped with learned reversal and color-switch maps. Closed-loop products become a consistency signal, allowing the model to retain direction-sensitive relational information that standard permutation-invariant message passing can lose.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Noncommutative Cluster Varieties and Moduli Spaces of Local Systems arXiv:2608.27284
✓✓ Beats tuned baseline 2026

Continued-Fraction Lacunary Features

Replace random Fourier features or a dense sinusoidal positional encoding with a compact bank whose frequencies are the continued-fraction denominators of an irrational number. Inverse-frequency amplitudes provide multiscale structure with a controlled sub-Lipschitz regularity profile, while lacunarity reduces the number of frequencies needed to represent oscillatory structure.

Useful6/10
Difficulty3/10
Novelty5/10
Paper: Regularity, quantitative deviation, and non-rigidity of a lacunary skew product arXiv:2608.25821
✗ Mechanism failed 2026

Wick-Matching Polynomial Interaction Layer

Replace an unconstrained high-order polynomial interaction module with features generated by Gaussian matrix contractions and their exact Wick expansion. The resulting interactions are sums of products of power-sum invariants, with coefficients fixed by perfect-matching counts, providing a low-parameter inductive bias for permutation- or orthogonal-structured data.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Stable Symmetric Series, Differential Operators, and Jack Deformations arXiv:2608.25651
△ Mechanism confirmed, baseline not beaten 2026

Harmonic Global Latent Channels

Augment a mesh or graph neural network with an explicit low-dimensional channel for topological circulation or flux modes. The network predicts a local gauge-fixed field u and global coefficients a, then reconstructs the physical field as y = u + Ha, so local message passing does not need to synthesize global modes through many layers.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Optimal Control in Hilbert Complex Spaces with Finite Element Exterior Calculus arXiv:2608.25266
✓✓ Beats tuned baseline 2026

Private spectral whitening front-end

Estimate the temporal spectrum of each sequence channel using a locally private procedure, then apply a regularized inverse-square-root spectral filter before the sequence enters attention or an SSM. The filter removes predictable low-frequency or narrow-band redundancy while avoiding unstable amplification at frequencies where the private estimate is small.

Useful6/10
Difficulty5/10
Novelty7/10
Paper: On the privacy cost for dependent Gaussian data: spectral density estimation under local differential privacy arXiv:2608.24847
Unverified 2026

Mixed-Subgroup Conservation Router

Replace an unconstrained Cartesian-product router over heterogeneous branches with a router whose joint expert or state assignments obey a finite-group conservation rule. Branch i emits a distribution over labels in its own subgroup H_i of a common finite abelian group G; only tuples whose group sum is zero are retained. This gives an exact, differentiable structural prior for modular arithmetic, multi-relational graphs, multi-view fusion, or any setting where latent labels compose by a…

Useful5/10
Difficulty5/10
Novelty7/10
Paper: An Affine Semigroup from Orbifold Boundary Conditions: cut, phylogenetic and hierarchical models in the unit-weight sector, and weighted configurations beyond them arXiv:2609.02630
Unverified 2026

Spherical covering initialization

Replace independently random unit-normalized prototypes in a spherical classifier, vector-quantizer, or prototype contrastive head with prototypes selected to cover the sphere evenly. The method directly targets the paper's finding that weak-signal spherical K-means preserves initialization-induced Voronoi structure, reducing redundant prototypes and making early assignments less dependent on random seed.

Useful5/10
Difficulty4/10
Novelty5/10
Paper: A Geometric Analysis of Initialization Bias in Spherical $K$-means in the Weak Signal Regime arXiv:2609.02205
Unverified 2026

Zero-mode MPS front-end for constrained sequence models

Build a constrained autoregressive model whose initial logits are generated from a translation-invariant MPS associated with a local zero-mode construction. The MPS supplies a structured valid distribution before a Transformer residual is added, so the model starts on the constraint manifold instead of learning validity through a penalty.

Useful5/10
Difficulty6/10
Novelty6/10
Paper: Efficient search for excitable zero-modes in constrained systems arXiv:2608.31165
Unverified 2026

Principal-bundle gauge-fixed Hamiltonian network

Represent a time-dependent Hamiltonian system on the reduced state $(q,t,p_q)$ rather than on the redundant extended state $(q,t,p_q,p_t)$. A neural Hamiltonian section predicts one canonical representative of each affine cotangent fiber, while an optional symmetry loss enforces consistency under transformations that translate time.

Useful5/10
Difficulty5/10
Novelty8/10
Paper: Reduction of symmetric time-dependent Hamiltonian systems I: presymplectic principal $\mathbb{R}$-bundles arXiv:2608.28278
Unverified 2026

Positive Lattice Fourier Features

Construct positional or relative-position features as a nonnegative mixture of lattice cosine functions instead of independently signed sinusoidal features. The resulting bias is the Fourier transform of a positive discrete measure with explicitly bounded spectral support, while the mesh and degree can be initialized in the paper's dense-but-controlled frequency regime.

Useful5/10
Difficulty4/10
Novelty7/10
Paper: Mesh-Degree Rigidity for Positive Chebyshev-Fourier Approximants arXiv:2608.28792
Unverified 2026

Crystal-Structured Discrete Latents

Use reverse plane partitions of a minuscule heap as the discrete codebook for a VQ-VAE or discrete sequence model. Codes are not arbitrary indices: each code is an order-preserving array, and crystal raising/lowering operators define a sparse, semantically structured neighborhood graph for augmentation, routing, and metric regularization.

Useful5/10
Difficulty5/10
Novelty8/10
Paper: Special Kirillov-Reshetikhin crystals arXiv:2608.27949
Unverified 2026

Zero-forcing causal lattice mixer

Build a sparse recurrent graph-neural layer on a path-by-path, path-by-cycle, or cycle-by-cycle latent lattice using a skew-zero-forcing seed set and its forcing order as a causal update schedule. Only the currently forced target node is activated at each step, so a small number of anchor states can propagate through the complete lattice while retaining local connectivity and periodic-boundary structure. The exact seed-count formulas predict the minimum number of anchors required by the graph…

Useful5/10
Difficulty6/10
Novelty6/10
Paper: Grundy Total Domination and Skew Zero Forcing in Cartesian Products of Paths and Cycles arXiv:2608.27804
Unverified 2026

Companion Observer Memory for Neural Policies

Replace an unrestricted GRU or attention-based history encoder with a fixed companion-form shift register driven by the current action and observation, followed by a learned nonlinear policy. The register stores a structured finite history, while a learned matrix or MLP readout maps that history to a control-relevant latent state. This should provide a cheaper and more interpretable memory mechanism for partially observed environments, especially when the relevant dynamics are approximately…

Useful5/10
Difficulty4/10
Novelty6/10
Paper: Data-Driven Output Feedback based Analysis and Control for Unknown Discrete-Time Linear System arXiv:2608.18452
Unverified 2026

Schur torsion-filter feature layer

Add a deterministic feature layer that evaluates symmetric Schur-type features on a fixed cyclic orbit and learned reciprocal latent pairs, then projects the resulting channels onto selected residue classes with an exact roots-of-unity filter. The reciprocal construction makes the layer invariant under replacing each latent scalar by its inverse, while the torsion projector prevents leakage between cyclic frequency sectors.

Useful5/10
Difficulty6/10
Novelty6/10
Paper: Schur polynomials twisted by roots of unity and reciprocal pairs: torsion filters, fusion quotients, and total unimodularity at odd order arXiv:2608.18302
Unverified 2026

Velocity-Scaled Symbolic Flow Model

Represent a continuous-time neural dynamical system as a symbolic Markov chain over regions together with a positive learned roof function giving the time spent in each region. Weight local reconstruction and prediction errors by the predicted vector-field speed, following the paper's scaled Hölder coding relation, so that the model does not over-penalize arbitrarily small coordinate errors near equilibria. This produces a hybrid latent model with discrete long-range structure and continuous…

Useful5/10
Difficulty6/10
Novelty7/10
Paper: Symbolic dynamics for non-uniformly hyperbolic flows arXiv:2608.14095
Unverified 2026

Independent-Set Neural Output Head

Replace an unconstrained categorical or multilabel output head with a graph-supported distribution over feasible independent sets. Given neural logits, assign probability proportional to the exponential of the total logit of each selected vertex, so incompatible vertices can never be jointly active. Use exact junction-tree inference for decomposable graphs with small treewidth, and compare against post-hoc masking or penalty-based constraint enforcement.

Useful5/10
Difficulty6/10
Novelty6/10
Paper: Graphical Models for Multivariate Count Data arXiv:2608.11366
Unverified 2026

Random-cluster anti-correlated routing

Replace independent Bernoulli branch dropout in a tree-structured mixture or hierarchical MLP with connectivity gates sampled from a q<1 wired random-cluster model. The q<1 law provides conditional negative association across branches, so increasing statistics of disjoint branches have nonpositive covariance; this should reduce redundant expert activation while preserving structured stochastic exploration.

Useful5/10
Difficulty6/10
Novelty6/10
Paper: The $q<1$ Random-Cluster Model on Wired Trees: Uniqueness and Negative Dependence arXiv:2608.08565
Unverified 2026

Proper binary edge positional encoding for tree GNNs

Precompute a two-valued edge labeling of every input tree so that adjacent vertices have different weighted incident-edge sums. Feed the edge labels and resulting vertex signatures into message passing as deterministic symmetry breakers. This can distinguish branches that otherwise produce identical initial representations without adding trainable parameters or random node identifiers.

Useful5/10
Difficulty4/10
Novelty7/10
Paper: Proper $\{a,b\}$-edge-weightings of trees arXiv:2608.08438
Unverified 2026

Adaptive Lambda-Quantile Prediction Head

Replace a fixed quantile output with a Lambda-quantile head that receives a predictive sample set and applies a learned value-dependent threshold \(\Lambda(x)\). Unlike ordinary quantile regression, the model can use a low threshold in one value range and a high threshold in another, which is useful when error costs or calibration requirements vary across the output domain. Start with a piecewise-constant or monotone spline parameterization, then test whether allowing controlled…

Useful5/10
Difficulty5/10
Novelty6/10
Paper: Lambda-quantiles under the microscope arXiv:2608.07122
Unverified 2026

Frame-safe totally-positive front-end

Replace the first learned one-dimensional convolution or STFT-like feature extractor with a differentiable bank of time-frequency shifts of a totally positive window. Parameterize the temporal spacing \(\alpha\) and frequency spacing \(\beta\) so that \(\alpha\beta<1\) is always satisfied, giving a mathematically certified oversampled representation instead of an arbitrarily subsampled filterbank.

Useful5/10
Difficulty5/10
Novelty5/10
Paper: Gabor Frames of Totally Positive Functions: A Complete Characterization arXiv:2608.04992
Unverified 2026

Injective Boundary-Aware Disk Pooling

Replace fixed-radius image blur or pooling with disk averages whose radius is proportional to the distance from each pixel to the image boundary. Compute the transform at every spatial location and train a lightweight decoder to reconstruct the pre-transform feature map, using reconstruction error as an anti-collapse regularizer. This creates a scale-adaptive smoothing layer with an injectivity motivation in the continuum while providing larger context in the image interior.

Useful5/10
Difficulty5/10
Novelty7/10
Paper: Variable-Radius Disk Transforms and an Area-Integral Problem of Zalcman arXiv:2608.02546
Unverified 2026

G2 structured latent code

Use the signed-base expansion as a compact discrete-continuous latent parameterization for a VAE or autoencoder. A short binary sequence produces exponentially refined coordinates, while a learned Markov prior captures correlations between successive latent bits. The decoder receives the resulting bounded real coordinates instead of an unconstrained Gaussian latent vector.

Useful5/10
Difficulty5/10
Novelty6/10
Paper: Fractal random variables defined by probability distributions of digits of their $G_2$-representation having two bases with different signs arXiv:2607.29327
Unverified 2026

Distributional spectral-preconditioned features

Replace or augment a singular scalar activation \(\sigma\) with a distributionally regularized activation \(g\) whose Fourier transform is multiplied by \((i\rho)^\alpha\). This suppresses the problematic low-frequency singular component and can produce better-conditioned random-feature or first-layer representations, while a residual raw-activation branch prevents loss of standard approximation behavior.

Useful5/10
Difficulty5/10
Novelty8/10
Paper: Radon Measure Representations for Infinite-Width Neural Networks with Singular Activations arXiv:2607.29258
Unverified 2026

Gradient-Commutator Neural Dynamics

Build a continuous-time neural dynamics module from scalar potential networks and their iterated Lie brackets instead of directly predicting an unrestricted vector field. Gradient primitives provide structured vector fields, while commutators add non-conservative and rotational directions; the paper proves that finite spans of such objects generate every smooth vector field on the stated compact manifold.

Useful5/10
Difficulty6/10
Novelty8/10
Paper: The Lie algebra generated by gradient vector fields arXiv:2607.26890