Unverified
2026
Replace a learned dense token-mixing matrix or residual-state transition with a sparse diffusive mixer whose Laplacian has a deliberately small largest Jordan block. Balance the two chain lengths around the central coupling/core, because the paper proves that this minimizes the worst defective transient among the tridiagonal family. Use a scalar residual step size to move the non-consensus spectrum inside the unit disk while preserving the sparse structure.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Construct an orthogonally equivariant residual map on symmetric feature matrices whose update is strongly monotone by adding the identity to a monotone isotropic tensor function. This provides a stability-controlled matrix block and a route to well-behaved inverse or fixed-point inference, rather than relying only on unconstrained residual weights.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace an unconstrained matrix nonlinearity on small symmetric feature blocks with the isotropic spectral lift of a permutation-equivariant monotone map on eigenvalues. The layer remains orthogonally equivariant, while the paper's equivalence transfers a scalar inner-product monotonicity certificate from eigenvalue space to the full matrix space.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Treat each attention head as a directed Markov graph and penalize token pairs that require many propagation steps to reach one another. This discourages isolated attention communities and slow information mixing while preserving the ordinary task objective.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Use enumerated weighing matrices as sparse orthogonal channel-mixing operators inside MLPs or residual blocks. Their ternary entries reduce multiplication to signed additions, while exact orthogonality prevents amplification or attenuation of feature norms; a trainable fixed-support version can recover expressivity without giving up computational sparsity.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Use the paper’s observation that the fully irreducible vertex is approximately local after crossed-channel ladders are removed to build a block-local curvature correction for neural-network optimization. Estimate a cheap bare covariance and subtract the inverse full covariance to obtain a local irreducible correction, avoiding a dense four-point model while retaining interaction effects that ordinary diagonal preconditioners miss.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Use the support graph of a sparse attention or routing mask as a feasibility object before applying normalization. Reject or minimally repair masks whose neighborhoods violate Hall's condition, ensuring that every selected query/key subset has enough reachable mass to support the desired marginals.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Represent a neural-network weight tensor by rank-one terms whose mode factors are selected from shared orthonormal bases, and impose the same basis alignment across tensor flattenings. During or after training, retain the largest coefficients to obtain a structured truncation analogous to truncated SVD. This should produce better-conditioned tensorized layers than unconstrained CP factors while preserving a directly controllable accuracy/compute tradeoff.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Use a strictly upper-triangular block operator to represent interactions between ordered layers, experts, or token groups, and compute its inverse exactly with a finite Neumann series. Because nilpotency truncates the series after a known number of blocks, the module avoids an iterative solver while retaining controlled long-range interactions.
Useful6/10
Difficulty5/10
Novelty8/10
Unverified
2026
Construct a neural layer with two independently ordered transformations and train its operators to satisfy the paper's diamond equations, so that applying direction 1 then direction 2 gives the same result as direction 2 then direction 1. Unlike ordinary weight sharing, the mixed identity permits noncommuting operators whose interaction defects cancel exactly.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Use the paper's monotone allocation viewpoint to decide which parameters receive progressively more pruning, bit reduction, or update opportunities. A van der Corput order prevents a fixed early prefix of structured groups from being systematically over-pruned, while the logarithmic comb bound quantifies worst cumulative imbalance across nested groups.
Useful6/10
Difficulty4/10
Novelty6/10
Unverified
2026
Project each FFN residual update onto the tangent space of the current token residual direction before adding it to the stream. This preserves the component that changes representation direction while suppressing norm-only motion, which may reduce residual-norm drift and aggregation-induced representation collapse.
Useful6/10
Difficulty4/10
Novelty6/10
Unverified
2026
Replace a learned sequence-mixing matrix with a structured lower-triangular Sprugnoli operator whose square is exactly the identity. Applying the same operator in reverse reconstructs activations exactly, so it can be used as a reversible Transformer mixer or reversible channel permutation while retaining nontrivial long-range mixing.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace raw pairwise attention or graph-edge scores by exact U-centered residuals, removing additive effects attributable to either endpoint. The resulting scores represent interaction beyond independent source and destination biases and satisfy zero row sums, preventing a few high-degree or high-activation tokens from dominating relational aggregation.
Useful6/10
Difficulty3/10
Novelty6/10
Unverified
2026
Replace a recursive product implementation of a rational spectral filter with an additive sum of independently evaluated resolvents. Use the layer on a graph Laplacian, token-similarity operator, or other sparse feature operator to obtain a high-order filter without multiplicative roundoff and gradient amplification; the independent solves can also be batched or distributed across devices.
Useful6/10
Difficulty6/10
Novelty6/10
Unverified
2026
Attach a predictor from each deeper representation to the representation immediately below it, and penalize the Gaussian KL divergence between the predicted lower-layer state and the actual lower-layer state. Learn or estimate one positive variance per layer so easy, low-noise layers receive high precision while intrinsically uncertain layers are not forced to fit their targets exactly.
Useful6/10
Difficulty4/10
Novelty5/10
Unverified
2026
Replace or augment a mixture-of-experts router with a relative transverse-curvature score computed between experts, rather than relying only on the router MLP logits. Experts that provide a broader, less stiff local response in task-relevant directions receive higher routing probability, while common nuisance or spectator directions cancel from the comparison. The score is invariant under a common linear reparameterization of the routing coordinates and can be restricted to a low-dimensional…
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Use the paper's independence condition as a design principle: prevent the gate's type-dependent admission behavior from being strongly correlated with downstream service cost. In an MoE or dynamic inference system, this discourages the gate from rejecting cheap requests and then preferentially admitting expensive requests when the queue happens to be shorter.
Useful6/10
Difficulty4/10
Novelty6/10
Unverified
2026
Use three learned state-transition operators corresponding to three data axes, and train them to satisfy the paper's pullback-style interchange rule. For every local pair of axes, two successive updates should reach the same square state; for triples of axes, all six update orders should agree. This reduces sensitivity to scan direction and limits long-horizon drift caused by inconsistent local transitions.
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace arithmetic averaging of feature covariances by the weighted Bures–Wasserstein barycenter of several SPD covariance matrices. The layer aggregates covariance statistics from augmentations, heads, channels, or local patches in a way that respects the geometry of centered Gaussian feature distributions and remains invariant under congruence changes of coordinates.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Track the dominant rank-r subspace of the gradient covariance online, then use that basis to construct a low-rank adaptive update or a controlled preconditioner. Unlike offline PCA refreshes, the Oja flow continuously follows changing training geometry while preserving orthonormality, potentially reducing the cost of second-order or Shampoo-like methods.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Construct a deep sequence model as a layered channel network with fixed random K-regular connections between neighboring depth layers, instead of dense or independently random weight matrices. Use norm-preserving edge normalization and a reversible residual update so that geometric randomness controls information transport while trainable nonlinear readouts provide task-specific computation. The architecture exposes a tunable crossover between quasi-one-dimensional ballistic or localized…
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Represent a token sequence or image region partition by a pair of planar rooted binary trees and a permutation of their leaves, following the tree-pair model of Thompson-type groups. Train the mixer to produce the same output whether a leaf is left intact or refined into two child leaves that are immediately merged by a matching inverse refinement. This creates a variable-resolution hierarchical module with an explicit consistency constraint instead of relying on an arbitrary fixed tree depth.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace a single graph or token-mixing operator with two coupled channels: an antisymmetric, coherence-preserving transport channel and a state-dependent dissipative diffusion channel. The local feature state controls the dissipative edge rates, so strongly occupied or conflicting regions are smoothed while weakly interacting regions retain rapid coherent propagation.
Useful6/10
Difficulty6/10
Novelty7/10