Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

Unverified 2026

Braid-word reversible mixer

Replace a dense token- or channel-mixing matrix with a product of local braid generators acting on adjacent coordinates. Each generator is an exactly invertible 2-by-2 transformation, while the braid and far-commutativity identities give multiple equivalent factorizations of the same global operator. This creates a sparse, reversible mixer with O(kn) cost for a braid word of length k, rather than O(n^2) cost for a dense matrix.

Useful5/10
Difficulty4/10
Novelty7/10
Paper: Fox $p$-Colorings as Fixed Points of Braid Representations arXiv:2608.29046
Unverified 2026

Divisibility-Weighted Simplicial Message Passing

Replace ordinary simplicial incidence matrices in a graph or mesh neural network by integer-ratio weighted incidences derived from a divisibility hierarchy on simplex weights. The resulting up/down message-passing operators preserve exact chain cancellation, so features propagated around a filled simplex cannot create spurious boundary signals. Train the weights either from known metadata or as positive integer powers of a small prime, while retaining an ordinary-incidence baseline for ablation.

Useful5/10
Difficulty5/10
Novelty6/10
Paper: Weighted Homology and Cohomology of Weighted Polyhedra arXiv:2608.29013
Unverified 2026

Banded Production-Matrix Polynomial Layer

Replace a dense learned polynomial-feature transform with a d-orthogonal recurrence whose production matrix is constrained to a (d+2)-banded lower-Hessenberg form. The layer generates successive features using only local recurrence coefficients, giving O(dN) arithmetic and O(dN) parameters for N basis functions instead of O(N^2) dense mixing.

Useful5/10
Difficulty5/10
Novelty7/10
Paper: Riordan array representation of recursive polynomial sequences, orthogonal polynomial sequences, and $d$-orthogonal polynomial sequences arXiv:2608.28834
Unverified 2026

Positive Lattice Fourier Features

Construct positional or relative-position features as a nonnegative mixture of lattice cosine functions instead of independently signed sinusoidal features. The resulting bias is the Fourier transform of a positive discrete measure with explicitly bounded spectral support, while the mesh and degree can be initialized in the paper's dense-but-controlled frequency regime.

Useful5/10
Difficulty4/10
Novelty7/10
Paper: Mesh-Degree Rigidity for Positive Chebyshev-Fourier Approximants arXiv:2608.28792
Unverified 2026

Matroidal Mahalanobis Attention

Parameterize a learned token metric as a nonnegative sum of sparse integral rank-one projections with unimodular support, rather than learning an unconstrained dense positive-semidefinite matrix. Graph-incidence covectors give an immediately implementable support family, while nonnegative coefficients guarantee positive semidefiniteness by construction.

Useful5/10
Difficulty5/10
Novelty5/10
Paper: Nonnegative conorms, regular matroids, and the tropical Schottky problem arXiv:2608.28783
Unverified 2026

Crystal-Structured Discrete Latents

Use reverse plane partitions of a minuscule heap as the discrete codebook for a VQ-VAE or discrete sequence model. Codes are not arbitrary indices: each code is an order-preserving array, and crystal raising/lowering operators define a sparse, semantically structured neighborhood graph for augmentation, routing, and metric regularization.

Useful5/10
Difficulty5/10
Novelty8/10
Paper: Special Kirillov-Reshetikhin crystals arXiv:2608.27949
Unverified 2026

Zero-forcing causal lattice mixer

Build a sparse recurrent graph-neural layer on a path-by-path, path-by-cycle, or cycle-by-cycle latent lattice using a skew-zero-forcing seed set and its forcing order as a causal update schedule. Only the currently forced target node is activated at each step, so a small number of anchor states can propagate through the complete lattice while retaining local connectivity and periodic-boundary structure. The exact seed-count formulas predict the minimum number of anchors required by the graph…

Useful5/10
Difficulty6/10
Novelty6/10
Paper: Grundy Total Domination and Skew Zero Forcing in Cartesian Products of Paths and Cycles arXiv:2608.27804
Unverified 2026

Coherence-aware superposition bottleneck

Insert an overcomplete sparse feature bottleneck into an MLP or embedding stream: encode an activation h with z = ReLU(W^T h + b), then reconstruct or continue computation from Wz. Normalize dictionary columns and train them to remain nearly tight and low-coherence, while choosing a negative bias from an estimate of worst-case cross-feature interference. The hypothesis is that this gives cleaner, more stable feature supports than an ordinary L1 sparse autoencoder at the same latent width.

Useful5/10
Difficulty5/10
Novelty4/10
Paper: Towards a mathematical theory of superposition arXiv:2608.27540
Unverified 2026

Transverse Fourier Collision Control

Construct a Fourier layer whose active frequencies lie on several nonparallel polygonal patches or thin annular sectors, and cap repeated difference vectors generated by pairs of patches. The bounded-multiplicity geometry limits how many input frequency pairs can contribute to the same output frequency, potentially reducing spectral aliasing and gradient variance in nonlinear Fourier mixing.

Useful5/10
Difficulty6/10
Novelty7/10
Paper: Quantitative Uniqueness and Rough Damping on $\mathbb T^2$ arXiv:2608.27544
Unverified 2026

Odd-Harmonic Spherical Positional Encoding

Replace or augment a scalar periodic positional coordinate with a normalized bank of odd Fourier harmonics, keeping every position on the same-radius sphere. The resulting representation has an explicit translation-invariant similarity kernel, allowing the frequency count and spectral weighting to control how sharply attention distinguishes nearby versus distant phases.

Useful5/10
Difficulty3/10
Novelty3/10
Paper: Tight upper bound on $d_{GH}(S^1,S^{2k+1})$: GPT's short proof arXiv:2608.21587
Unverified 2026

Fractal surface-to-volume coordinate router

Build a fixed multiscale router that maps 2D coordinates to 3D voxel coordinates using the paper's X-shaped self-similar refinement. Use the router to run a 3D feature field or volumetric token mixer over a 2D-organized tensor, while retaining a mathematically controlled locality bound instead of an arbitrary flattening permutation. The first target is a 3D neural field or small voxel classifier where the router replaces either a dense 3D feature table or a naive raster-order token layout.

Useful5/10
Difficulty6/10
Novelty6/10
Paper: Space-filling surfaces: sharp Hölder continuous parameterizations from squares to cubes arXiv:2608.21246
Unverified 2026

Lattice Monodromy Residual Block

Insert a fixed reversible lattice shear into a residual network so successive blocks follow a structured monodromy orbit rather than using unrelated learned transformations. Apply the transformation to a small learned subspace of hidden channels while leaving the remaining channels unchanged. This creates deterministic phase-dependent feature mixing with no additional trainable parameters.

Useful5/10
Difficulty5/10
Novelty8/10
Paper: $G_2$-Manifolds from 4d $\mathcal{N}=1$ Quivers arXiv:2608.21238
Unverified 2026

Jumbled sparse attention masks

Design sparse attention masks using a graph discrepancy criterion rather than selecting only local or nearest-neighbor edges. A mask with approximately uniform edge counts between every pair of token subsets spreads information globally, while the rigidity consequence provides a principled way to preserve enough independent pairwise constraints for latent geometric features.

Useful5/10
Difficulty4/10
Novelty6/10
Paper: Rigidity of expanders and pseudorandom graphs arXiv:2608.21058
Unverified 2026

Scalene anticommuting three-token mixer

Replace an unconstrained three-token interaction block by three distinct pair maps constructed from anticommuting channel generators. For every token triple, enforce equality of the two composition paths A12 B13 C23 and C23 B13 A12, while retaining different parameters for the three edges. This creates a globally consistent three-way interaction without collapsing to a single shared pair operator.

Useful5/10
Difficulty6/10
Novelty9/10
Paper: Multiparameter Quantum Affine Spaces and the Scalene Yang--Baxter Equation arXiv:2608.20714
Unverified 2026

Adjoint spectral opening for stable pooling

Construct a learnable spectral pooling block as an erosion followed by its adjoint dilation, making the resulting opening idempotent, increasing, and anti-extensive. The block can suppress frequencies outside a learned passband while guaranteeing that applying it twice does not continue changing the representation, which is useful in multi-stage CNN pyramids and U-Net skip paths.

Useful5/10
Difficulty6/10
Novelty6/10
Paper: Morphological Representation Theory in the Fourier Inf-Semilattice: Universal Decomposition of Frequency-Domain Deep Learning Operators arXiv:2608.20399
Unverified 2026

RG-monotone representation compression

Interpret successive neural representations as an RG flow and constrain coarse-graining layers to remove unstable or redundant information monotonically. The paper reports monotonic decrease of an effective central charge along measurement-induced RG flows; a neural analogue can use a measurable information-complexity proxy and reject compression steps that increase it while preserving task-relevant information.

Useful5/10
Difficulty6/10
Novelty6/10
Paper: Learning Potts Models and $Z_3$ Toric Codes: Higher and Ordinary Nishimori Criticality arXiv:2608.20268
Unverified 2026

Convex-order stochastic expert layer

Replace a deterministic mixture-of-experts residual block with K population-indexed stochastic expert states coupled through a graphon matrix. The layer uses a shared drift and expert-dependent diffusion, while an empirical convex-order penalty makes later representations more dispersed than a reference representation without permitting a mean shift.

Useful5/10
Difficulty5/10
Novelty7/10
Paper: Convex order preservation for graphon mean-field systems arXiv:2608.19576
Unverified 2026

Holonomy-composed latent transformations

Replace unconstrained transformation composition in a geometric or sequence encoder with time-dependent Lie-algebra controls whose flows compose according to the paper's flow-product rule. Add a holonomy consistency loss so different control trajectories that induce the same endpoint automorphism produce the same latent transformation, reducing sensitivity to arbitrary path parameterization.

Useful5/10
Difficulty6/10
Novelty7/10
Paper: On the holonomy of Lie algebroids arXiv:2608.19399
Unverified 2026

Budgeted Random Tree Attention

Replace dense attention on tree-structured inputs with stochastic attention neighborhoods formed by metric balls of sampled radii. Use the paper's exact trimming rule to ensure that every sampled cover remains valid while its total radius budget is bounded, then average predictions over several independent covers during training. This creates sparse, globally covering attention masks with an explicit locality-versus-coverage control.

Useful5/10
Difficulty6/10
Novelty8/10
Paper: Exact random covers of metric trees: balanced rounding, duality, and sharp thresholds arXiv:2608.18967
Unverified 2026

Companion Observer Memory for Neural Policies

Replace an unrestricted GRU or attention-based history encoder with a fixed companion-form shift register driven by the current action and observation, followed by a learned nonlinear policy. The register stores a structured finite history, while a learned matrix or MLP readout maps that history to a control-relevant latent state. This should provide a cheaper and more interpretable memory mechanism for partially observed environments, especially when the relevant dynamics are approximately…

Useful5/10
Difficulty4/10
Novelty6/10
Paper: Data-Driven Output Feedback based Analysis and Control for Unknown Discrete-Time Linear System arXiv:2608.18452
Unverified 2026

Beta-fragmented hierarchical attention

Build a binary hierarchy over tokens by recursively splitting each active block with a beta-splitting rule, then perform dense attention only inside small leaf blocks and communicate between leaves through learned summaries at internal nodes. The beta parameter controls how balanced the partition is, while the paper's maximum-depth asymptotic supplies a principled depth budget and a way to detect pathological trees.

Useful5/10
Difficulty6/10
Novelty5/10
Paper: Asymptotics for Beta-Splitting Trees via Homogeneous Fragmentations and Meromorphic Potential Theory arXiv:2608.18320
Unverified 2026

Schur torsion-filter feature layer

Add a deterministic feature layer that evaluates symmetric Schur-type features on a fixed cyclic orbit and learned reciprocal latent pairs, then projects the resulting channels onto selected residue classes with an exact roots-of-unity filter. The reciprocal construction makes the layer invariant under replacing each latent scalar by its inverse, while the torsion projector prevents leakage between cyclic frequency sectors.

Useful5/10
Difficulty6/10
Novelty6/10
Paper: Schur polynomials twisted by roots of unity and reciprocal pairs: torsion filters, fusion quotients, and total unimodularity at odd order arXiv:2608.18302
Unverified 2026

Veronese Projective Feature Layer

Replace or augment the first embedding layer for antipodally identified inputs with the normalized traceless quadratic map from the Veronese construction. Because q and -q produce exactly the same feature, the layer enforces projective invariance by construction rather than learning it from augmented examples. The resulting matrix-valued features can be flattened, projected, or processed by an equivariant linear layer.

Useful5/10
Difficulty2/10
Novelty6/10
Paper: Normal Curvature and the Projective Systole arXiv:2608.18002
Unverified 2026

Potential-weighted fractional diffusion layer

Insert a positivity-preserving fractional Schrödinger resolvent into a 1D neural sequence block. Given a nonnegative learned potential V, the layer transforms an input signal f using V^a(-Delta+V)^(-a)f, allowing the network to learn where to smooth or suppress features while retaining an L1 bound independent of the potential magnitude. Use a in (0,1] as a fixed hyperparameter or a clipped learned scalar.

Useful5/10
Difficulty6/10
Novelty7/10
Paper: Potential-free $L^1$-estimates for positivity-preserving Riesz transform related to Schrödinger operator in dimension one arXiv:2608.17406