✗ Mechanism failed
2026
Replace Lorentz-hyperboloid tensors with proper-velocity tensors whose spatial coordinates can be transformed by standard Euclidean affine layers and activations. Reconstruct the Lorentz time coordinate only at manifold boundaries, preserving the hyperbolic representation while avoiding repeated projection, normalization, or fragile exponential-map calculations.
Useful7/10
Difficulty4/10
Novelty6/10
✗ Mechanism failed
2026
Replace a globally chosen constant learning rate with a blockwise rate calibrated to the local flatness exponent of the objective. If the local Hessian decays like \(\|x-x_\star\|^{m-2}\), choose the rate so that the predicted stationary parameter radius \(\alpha^{1/m}\) matches a prescribed exploration or optimization radius, rather than incorrectly using the quadratic rule \(\sqrt{\alpha}\).
Useful7/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Use the extreme-eigenvector marginal test to decide whether a Kronecker preconditioner is condition-optimal, rather than blindly running expensive factor refinement. If the certificate fails, construct a low-cost factor correction from the mismatch between tensor marginals of the worst-conditioned spectral states and accept it only with a condition-number line search.
Useful7/10
Difficulty7/10
Novelty8/10
△ Mechanism confirmed, baseline not beaten
2026
Build a low-dimensional neural-network geometry from trainable observables or probes instead of estimating the full Fisher matrix. Precondition the parameter gradient by the inverse variability of the probes and their parameter sensitivity, producing a task-adapted update that can remain usable for implicit models, heavy-tailed data, and parameter-dependent-support distributions.
Useful7/10
Difficulty6/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Route tokens to experts using Laguerre cells defined by the minimum control energy needed to move a token embedding to each expert prototype, rather than by Euclidean distance or an unconstrained learned router logit. Per-expert dual weights deform the cells so that minibatch routing follows prescribed expert capacities, giving a geometrically interpretable alternative to auxiliary load-balancing losses.
Useful7/10
Difficulty6/10
Novelty6/10
✗ Failed on benchmark
2026
Replace ordinary pairwise attention similarity by an affinity averaged over transformed keys or values. The resulting attention is invariant to the group action on either input and avoids requiring the network to learn identical attention patterns for every rotated or transformed copy.
Useful7/10
Difficulty5/10
Novelty5/10
△ Mechanism confirmed, baseline not beaten
2026
Replace a conventional two-layer channel mixer in a reversible architecture with the tropicalization of two cluster mutations. For every pair of channels, the block applies sign-dependent integer shears and reflections, giving a cheap piecewise-linear transformation that is exactly invertible and requires no stored activations during backpropagation. Continuous trainable affine scale and mixing parameters can be placed around the fixed tropical core.
Useful7/10
Difficulty4/10
Novelty8/10
△ Mechanism confirmed, baseline not beaten
2026
Build a multi-expert or multi-task layer whose feature channels are divided into a globally shared subspace and expert-private subspaces. Matrix-weighted message passing couples experts only through selected feature directions, while the nullspace preserves specialization; the graph-cut condition provides a concrete test that the shared channels can propagate across all experts rather than becoming disconnected islands.
Useful7/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Replace dense coarse-to-fine cross-attention at multiresolution interfaces with a sparse, nonnegative overlap operator whose weighted feature average is exactly conserved between the two resolutions. Use this operator as a low-order path and blend it with an unrestricted neural cross-attention path through a convex limiter that keeps features inside a box or simplex domain. The construction is especially suitable for adaptive token grids, hierarchical graph neural networks, neural operators…
Useful7/10
Difficulty5/10
Novelty8/10
✗ Mechanism failed
2026
Replace a learned binary MoE gate with a hyperplane whose two sides contain prescribed proportions of several token populations simultaneously. In a low-dimensional routing projection, solve the cap-volume equations directly, producing deterministic per-population load control without an auxiliary load-balancing loss. Recursively applying the construction yields a balanced binary expert tree.
Useful7/10
Difficulty6/10
Novelty7/10
Unverified
2026
Propagate uncertain inputs or parameter perturbations through a neural network with zonotopes, and choose the reduction rule after each layer using short lookahead that predicts downstream certified-bound error. Candidate reducers can include largest-generator retention, box conversion, and norm-based merging; unlike a fixed policy, the chosen rule depends on the current generator geometry and remaining network depth. The goal is tighter robustness bounds at the same generator budget, or the…
Useful6/10
Difficulty6/10
Novelty7/10
Unverified
2026
Add a consistency loss for a neural model processing irregularly sampled functions or sequences, comparing its latent covariance representation before and after a time-coordinate warp. The target discrepancy should be near zero for affine transformations but should grow with the normalized curvature, bandwidth, and local sparsity of a nonlinear warp.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Insert a projection-space mixer that combines several fixed or learned directions using reciprocal correlations with the current feature, then normalize the result. The exact construction has a universal beta law for its squared input-output cosine, so it can create controlled angular diversity while remaining deterministic and independent of the chosen direction dictionary.
Useful6/10
Difficulty5/10
Novelty8/10
Unverified
2026
Replace an unconstrained hidden-to-hidden interaction in an MLP or transformer feed-forward block by two gauge-related branches. Split channels with an orthogonal involution Θ, constrain the learned interaction K to anticommute with Θ, and use opposite signs of K in paired branches. This creates a testable inductive bias in which the learned interaction only transfers information between the two channel subspaces.
Useful6/10
Difficulty5/10
Novelty8/10
Unverified
2026
Represent each token value in an attention head as a bounded three-dimensional Lie-algebra vector and impose a weighted polygon-closure condition on the aggregate value vectors. The module is invariant to a common \(SU(2)\simeq SO(3)\) rotation of all token vectors, preventing the head from spending capacity on an arbitrary global orientation. A soft closure penalty gives a drop-in experiment, while projection onto the zero-sum manifold provides a harder constrained variant.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Add a hybrid geometric penalty between an attention distribution at one layer or training step and a reference distribution, such as detached attention from the preceding layer or optimization step. The penalty allows attention mass to move between nearby token positions at a transport cost while separately charging for local compositional changes, producing a structured alternative to KL or entropy regularization.
Useful6/10
Difficulty7/10
Novelty7/10
Unverified
2026
Replace a global attention truncation rule with a per-query local-tail budget. For each query token, retain nearby or high-priority keys until the estimated discarded interaction strength is below a target epsilon; this uses the paper's central distinction between local tail mass and the extensive norm of the discarded operator. The resulting attention pattern can allocate long-range computation only to tokens whose local tail is large.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Replace a neural network head that predicts a covariance or other SPD matrix entrywise with regression in the matrix-log domain. The network predicts a symmetric matrix in unconstrained Euclidean coordinates, the matrix exponential guarantees an SPD output, and training can use intrinsic log-Euclidean or affine-invariant errors rather than Frobenius error on raw entries.
Useful6/10
Difficulty4/10
Novelty5/10
Unverified
2026
Add a multiscale penalty to transformer token-mixing activations when they are simultaneously concentrated on a spatial or token subset and on a separated, irregular frequency subset. The penalty uses the fractal uncertainty scaling law to discourage hidden states from collapsing onto narrow token patterns and narrow spectral bands, potentially improving robustness to token masking and frequency-corrupted inputs.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Insert a hyperspherical adapter that splits an embedding into several unit-sphere blocks, changes the dimension of each block, and recombines them with a synchronized spherical join. Train the adapter to preserve pairwise angular distances, while using the paper's max-distortion composition principle to avoid uncontrolled accumulation of blockwise errors.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace ordinary absolute positional embeddings with coordinates on a learned flat torus and use dual-lattice Fourier characters as positional features. Control the covariance of the coordinate fundamental domain so that the paper's inequality guarantees a lower bound on the smallest nonzero positional frequency, preventing the learned periodic coordinate system from developing arbitrarily weak or nearly constant modes.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Use a fixed spherical t-design as the direction codebook for a directional attention or feature-aggregation module instead of independently sampled random directions. Equal weights provide exact zero mean and isotropic second moments, while exactness for spherical polynomials up to degree t reduces directional aliasing and seed-dependent anisotropy.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Add a neural feature, adapter, or expert block only through the component of its outputs that is orthogonal to the span of all previously installed blocks. Quotient coefficient directions that produce nearly identical outputs with an SVD or pseudoinverse, so the new block contributes intrinsic representational dimensions instead of duplicating old features. The expected benefit is a smaller effective architecture and better-conditioned block expansion.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Construct an orthogonally equivariant residual map on symmetric feature matrices whose update is strongly monotone by adding the identity to a monotone isotropic tensor function. This provides a stability-controlled matrix block and a route to well-behaved inverse or fixed-point inference, rather than relying only on unconstrained residual weights.
Useful6/10
Difficulty6/10
Novelty7/10