Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.
Replace Cox-de Boor evaluation of each cubic B-spline edge activation with its fixed truncated-power expansion. Normalize each scalar edge input to a bounded knot coordinate, evaluate the five shifted cubic positive-part terms in parallel, and contract them with the learned spline coefficients inside one fused kernel.
Replace ordinary topology-sensitive message passing with scalar-gated aggregation followed by an explicit correction that aligns local node states with a graph-wide consensus component. The correction should make node embeddings less sensitive to line or edge removals while preserving local information needed for prediction. This is suitable for graph neural networks and graph-based world models exposed to changing graph sizes or sparsity patterns.
Replace an unconstrained stochastic transition between categorical or discretized latent distributions by a transition matrix that preserves a prescribed reference distribution while mapping relative populations through a martingale. This prevents the layer from inventing arbitrarily sharp deviations from the reference and imposes a convex-order monotonicity condition on uncertainty across layers or diffusion time steps.
Construct a spatial recurrent network whose local vector hidden state has two stable attractors and whose neighbor coupling is diffusive. Train or constrain the network so that the desired attractor invades the undesired one with a controlled positive front velocity, rather than relying on a scalar class-frequency variable that can erase depletion and interface structure.
Attach a hard control-barrier-function quadratic-program safety filter to a neural policy, but solve the filter with operator splitting and differentiate through its fixed-point map using projection Jacobian-vector products. The network learns the nominal action and task objective end to end, while the deployed action remains the feasible filtered action rather than an unconstrained penalty-based approximation.
Replace direct learning of a highly cancelling signed observable with a quotient-space model over symmetry orbits of inputs. Predict a physically constrained baseline for each family and use an LSTM or set/graph encoder only for the residual many-body correlation, then aggregate family predictions with known signed weights instead of forming a noisy sample-level ratio.
Construct a mode-mixing layer from a few sphere-preserving vector fields and shift operators rather than a dense learned Fourier convolution. A commutator of two low-complexity shifted rotations produces a new interaction at the sum of their offsets, allowing long-range or high-frequency mode coupling to be synthesized with only a small number of primitive operators. The layer can be used whenever each feature vector is normalized to the sphere, or more generally as a norm-preserving block on…
Give a single spiking layer a persistent vector-valued apical compartment that stores the current online linear predictor for the task. On each labeled context pair, its subthreshold state performs a leaky LMS update; on the query, the state is read without updating, allowing in-context adaptation without attention or inference-time synaptic plasticity.
Replace an iid local or randomly sparse residual mixer with a distance-decaying long-range operator whose edge magnitudes are correlated through a shared latent Gaussian field. The paper predicts that these correlations qualitatively change low-energy spectral scaling and increase multiscale information propagation relative to iid long-range weights. Apply the operator as a spectrally normalized residual block so that the benefit comes from correlated scale coverage rather than uncontrolled…
Replace a fragile learned similarity score for continuous object coordinates with a projection residual against a learned signal subspace. Candidate coordinates are represented by normalized Fourier or positional feature vectors, and the score is the fraction of feature energy outside the estimated subspace. The score remains useful even when the estimated subspace is adversarially rotated, because the perturbation is controlled directly by a sine-theta distance rather than by assumptions about…
Use a spherical-design codebook and the paper's polar slack factorization to create a nonnegative geometric interaction bias for attention or expert routing. The resulting kernel is generated by a rank-one term and a rank-at-most-d term, and entries close to zero can define a structured sparse mask instead of relying only on learned top-k selection.
Use a local chaining complexity computed from an empirical input metric to predict stochastic output error for each individual input. Easy, locally concentrated inputs can use fewer dropout, ensemble, or diffusion samples, while high-complexity inputs receive additional computation; unlike a global confidence threshold, the allocation varies with the input.
Replace fixed random Fourier features in a coordinate network or PINN with a small complex Fourier dictionary whose propagation directions are learned from a weighted residual. Given directions, solve the linear feature coefficients exactly or by ridge regression, and optimize only the directions in the outer loop. This should represent low-directional-complexity fields with fewer features and avoid wasting gradient updates on coefficients that can be fitted analytically.
Attach a deterministic supervisory automaton to a neural policy or sequence model and mask every event disabled by the current supervisor state. Use a short receding-horizon planner over admissible events to resolve conflicts between neural preferences and shared-resource constraints. The network scores useful actions, while the automaton supplies an exact safety layer.
Make a neural network predict a vector potential rather than a magnetic or velocity field, then obtain the physical vector field with a fixed differentiable discrete curl. The reconstructed field satisfies the discrete divergence-free constraint exactly, eliminating divergence-penalty tuning and preventing constraint drift during long rollouts.
Make a neural network predict a positive Gaussian-mixture representation of the distribution function rather than independent values on a momentum grid. Use the mixture parameters inside a differentiable Boltzmann collision operator, so training directly enforces the interaction mechanism and exposes the relaxation spectrum responsible for ballistic-to-hydrodynamic crossover.
Evaluate weak residuals against a bank of periodic trigonometric test functions using FFT projections instead of repeated pointwise quadrature or output automatic differentiation. Frequency truncation and mode weighting provide a direct way to control the spatial scales enforced during neural PDE training.
Construct a recurrent or state-space layer as a skew product: an expanding bounded feature coordinate drives a linearly contracting hidden state. Constrain the hidden transition matrix A to have spectral radius below one, and monitor the predicted transition ell times the absolute determinant of A equals one: below it, hidden trajectories should occupy a thin or fractal set, while above it they should have substantially higher-dimensional state coverage without losing local contraction.
Replace or augment a recurrent hidden coordinate with a nonnegative bistable autocatalytic state driven by an external control signal. The cell retains information through metastable low and high states, while a periodic or slowly varying control produces a controlled phase lag and hysteresis useful for temporal regime detection. Explicit noise can be injected to test whether it enhances switching near the predicted intermediate-frequency regime.
Replace a single polynomial graph filter or dense inverse with a multiscale layer that applies unitary propagations at dyadic times and combines them according to the dyadic resolvent identity. For a symmetric graph operator, this implements a stable rational spectral filter that can selectively retain or suppress frequency bands while exposing logarithmic multiscale structure.
Replace Lorentz-hyperboloid tensors with proper-velocity tensors whose spatial coordinates can be transformed by standard Euclidean affine layers and activations. Reconstruct the Lorentz time coordinate only at manifold boundaries, preserving the hyperbolic representation while avoiding repeated projection, normalization, or fragile exponential-map calculations.
Replace a large diagonalizable recurrent or state-space transition operator by a sparse set of retained oscillatory modes selected according to their contribution to the output autocorrelation. Unlike magnitude-based pruning, the objective is to preserve the power-law return signal generated by pairwise spectral differences, enabling long memory with far fewer modes.
Use the KS ratio to decide how many message-passing layers to execute per graph or per node, rather than selecting a fixed depth. In the subcritical regime, stop once the predicted remaining effect is below a tolerance; in the supercritical regime, continue until the observed logit change becomes small or a larger budget is reached.
Run the same neural decoder over several algebraically equivalent augmented graphs and aggregate their variable-level predictions. Each graph exposes different cycle structure and message routes, providing structured architectural diversity rather than ordinary random-seed ensembling.