Machine-learning ideas tagged Inference speedup in the ML taxonomy of the Math2NN corpus.
Replace real diagonal state-space channels with complex damped oscillators whose hidden states encode both amplitude and phase. Train with parallel causal convolution and deploy with the equivalent one-step recurrence, allowing the same layer to support efficient batched training and low-memory streaming inference.
Replace top-k token pruning by greedy maximization of a diversity-aware monotone submodular utility under a spacing or coverage constraint. The selector repeatedly chooses the feasible token with the largest marginal utility, avoiding the redundant-token failure mode of independent score ranking while inheriting a constant-factor approximation guarantee under the stated 2-system abstraction.
Add a small constrained equilibrium layer whose response depends on the current neural state and recent exogenous history, then cache responses keyed by a learned history embedding. For a new history, reuse a cached response only when an empirical Wasserstein distance to the cached history is below a threshold; otherwise run a few inner optimization iterations. The paper's local Holder and trajectory-stability results motivate graceful degradation rather than catastrophic errors for nearby…
Replace a fixed maximum-wait timeout in dynamic neural inference batching with a service-aware timer. The timer estimates the marginal reduction in computation or communication cost from adding another request and delays service only while that saving justifies the additional oldest-request delay.
Replace an unconstrained spatial gate or interpolation kernel by a compactly supported function whose translates under a lattice exactly sum to one. Impose zero products between translates under a second lattice, so active gates do not collide; thresholding a positive superlevel set then provides a nonzero separation margin and predictable sparse computation.
Represent periodic input-output behavior using a compact real vector of Fourier coefficients and learn an invertible neural map from input coefficients to output coefficients. Inference then obtains the input representation for a desired periodic output by a single inverse pass instead of iterative optimization through a nonlinear forward model, while the Fourier representation reduces sequence dimensionality when high-rate signals are spectrally sparse.
Add an asynchronous binary refinement module in which each spatial unit or graph node may change its predicted label once if its current label disagrees with a weighted neighborhood field, after which it is permanently frozen. This prevents recurrent flip-flopping in iterative segmentation or denoising and should preserve large-scale structures while allowing a final interface-localized correction phase.
Replace repeated full-dimensional matrix-exponential or ODE solves in a conditioned continuous-time state-space layer with contour quadrature evaluated in a projection basis. The same reduced basis and contour nodes can serve many conditioning vectors, while shifted reduced resolvents provide a stable and differentiable approximation over a prescribed time window.
Use attention-graph hitting times to identify tokens whose information has not mixed through the network, then route only those tokens through additional Transformer blocks. Tokens with fast reachability exit early, while slow or isolated tokens receive more computation.
Use enumerated weighing matrices as sparse orthogonal channel-mixing operators inside MLPs or residual blocks. Their ternary entries reduce multiplication to signed additions, while exact orthogonality prevents amplification or attenuation of feature norms; a trainable fixed-support version can recover expressivity without giving up computational sparsity.
Use a strictly upper-triangular block operator to represent interactions between ordered layers, experts, or token groups, and compute its inverse exactly with a finite Neumann series. Because nilpotency truncates the series after a known number of blocks, the module avoids an iterative solver while retaining controlled long-range interactions.
Build a recurrent ReLU module that evaluates a binary refinement cascade using a fixed-dimensional state and shared cell weights. Replace hard binary digit selection with two overlapping circle coordinates; switch between their affine state updates only at points where the two candidate updates agree, so the switch is an exact continuous piecewise-linear ReLU operation rather than a multiplicative gate.
Replace a recursive product implementation of a rational spectral filter with an additive sum of independently evaluated resolvents. Use the layer on a graph Laplacian, token-similarity operator, or other sparse feature operator to obtain a high-order filter without multiplicative roundoff and gradient amplification; the independent solves can also be batched or distributed across devices.
Construct a symmetric token affinity matrix and approximate only its spectrally outlying token-mixing modes using a small set of sampled landmark columns. The resulting low-rank operator replaces an \(O(n^2)\) dense mixer by two skinny matrix multiplications, while the paper's residual guarantee predicts that large-magnitude global interaction modes are preserved.
Replace dense attention or strictly local windows with a refreshed sparse communication graph whose edge lengths follow a truncated power law. For sequence positions or image patches at distance $r$, sample edges with probability proportional to $(1+r)^{-(d+\alpha)}$; in the regime $0<\alpha<d$, repeated layers should spread information across the sequence much faster than a local window while using only $O(nk)$ edges per layer. The main test is whether this preserves long-range accuracy at…
Replace dense spatial attention on a 2D feature grid by attention from every non-landmark token to a small periodic set of landmark tokens. Choose landmarks using the paper's residue construction so that every non-landmark token has at least one landmark exactly k Manhattan steps away. Use this guaranteed edge as a sparse global-information route, combined with a cheap local convolution or residual path for accuracy.
Construct one reference Lanczos basis for a symmetric propagation operator H, then derive several seed-specific spectral responses for Q_a(H)x_0 through finite-band polynomial connectors. With degree-r seeds, each transformed basis vector uses at most 2r+1 neighboring reference basis vectors, avoiding a separate Lanczos factorization for every seed.
Use the paper's independence condition as a design principle: prevent the gate's type-dependent admission behavior from being strongly correlated with downstream service cost. In an MoE or dynamic inference system, this discourages the gate from rejecting cheap requests and then preferentially admitting expensive requests when the queue happens to be shorter.
Replace an SVD-based pseudoinverse of a learned rectangular matrix with a low-degree polynomial initialization followed by a few Newton–Schulz iterations. The polynomial approximates the inverse Gram operator, while a cheap residual test accepts it only when the iteration is contractive and otherwise selects a conservative transpose-scaled initialization.
Use normalized scheduling variables and explicitly cap the degree of their products in a neural LPV or mixture-of-dynamics model. Instead of allowing every multiplicative interaction between scheduling coordinates and past or future features, retain only monomials below a chosen degree threshold. This produces a controllable approximation knob between a purely linear model and a full lifted predictor, while avoiding unstable extrapolation caused by poorly scaled high-degree features.
Represent a token sequence or image region partition by a pair of planar rooted binary trees and a permutation of their leaves, following the tree-pair model of Thompson-type groups. Train the mixer to produce the same output whether a leaf is left intact or refined into two child leaves that are immediately merged by a matching inverse refinement. This creates a variable-resolution hierarchical module with an explicit consistency constraint instead of relying on an arbitrary fixed tree depth.
Build a sparse expert cascade whose router uses empirical progress, uncertainty, and compute cost to construct an efficient frontier of experts. Instead of always choosing the expert with the largest immediate gain per FLOP, route different confidence states through an ordered sequence of frontier experts, allowing cheap high-variance experts early and safer or more cost-effective experts near the final decision.
Replace direct source-to-target attention connectivity by two sparse incidence relations through a set of latent witness tokens. A source token attends only to a small set of witnesses, and each witness attends only to a small set of target tokens; the composed relation is trained to contain exactly one witness for desired pairs and no witnesses for undesired pairs. This produces a controllable sparse attention pattern whose errors can be measured entrywise against a dense teacher or known mask.
Replace uniform or random landmark selection in low-rank attention with approximate greedy pivoted QR selection on the attention matrix. At each step, select the key whose residual attention column has the largest norm, orthogonalize it against previously selected columns, and stop when the residual energy falls below a target tolerance. Use the selected columns as an adaptive attention basis and reconstruct the full attention output through a small coefficient matrix.