ML: Inference speedup

Machine-learning ideas tagged Inference speedup in the ML taxonomy of the Math2NN corpus.

✓✓ Beats tuned baseline 2026

Path-work correction for exact neural proposals

Use the learned path only as a global proposal, then correct complete trajectories rather than endpoints. Exponentiated negative work gives self-normalized importance weights, while the same path ratio gives an independent Metropolis acceptance probability. This turns an imperfect neural sampler into an asymptotically exact sampler whenever forward and reverse path laws overlap.

Useful8/10
Difficulty4/10
Novelty6/10
Paper: Neural Non-Equilibrium Hamiltonian Monte Carlo for Corrected Boltzmann Sampling arXiv:2607.15682
△ Mechanism confirmed, baseline not beaten 2026

Derivative-Free Very-Weak Neural PDE Solver

Train a neural trial function for an elliptic PDE using a very-weak residual in which all derivatives act on fixed smooth test functions rather than on the neural network. This eliminates second-order reverse-mode or forward-mode automatic differentiation and allows low-regularity activations while retaining a least-squares objective over many test functions.

Useful8/10
Difficulty4/10
Novelty6/10
Paper: Neural Very Weak Formulations enabling Hardware-Oriented deep PDE solvers arXiv:2607.14498
△ Mechanism confirmed, baseline not beaten 2026

Tail-Aware Verifier Portfolio

Use the paper's tail comparison to decide when another call from the same verifier family is useless and when to switch to a different model, modality, or evidence source. The objective is to reduce the high-alpha survivor population—the incorrect examples that consistently fool one verifier—rather than maximizing average one-shot verifier accuracy.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Partially Correlated Verifier Cascades in LLM Harnesses: Concave Log-Odds, Polynomial Reliability, and Blind-Spot Ceilings arXiv:2607.13918
△ Mechanism confirmed, baseline not beaten 2026

Biclique-Hub Attention

Replace a dense directed attention matrix by a collection of K learned source-to-hub-to-target interactions. Each hub corresponds to a directed biclique, allowing many source tokens to communicate with many target tokens using O(NK) rather than O(N^2) pair interactions. The construction preserves asymmetric information flow and can be initialized from a graph cover of high-attention edges.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: On Transformer Dynamics arXiv:2607.13295
✓✓ Beats tuned baseline 2026

Low-rank one-shot horizon predictor

Replace an autoregressive rollout of a learned dynamical model with a branch-trunk factorization that predicts all future steps simultaneously. The branch network encodes the future action sequence, while the trunk network encodes the current state and query coordinates; their inner products produce the complete horizon. This removes repeated state updates during inference and gives a compact differentiable model for planning.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: Model predictive control for laser thermal processing: operator learning, closed-loop validation, and out-of-distribution analysis arXiv:2607.13289
✗ Failed on benchmark 2026

Audited Risk-Budgeted Early Exit

Attach a cheap risk score to each neural-network prediction and skip an expensive verifier, ensemble, diffusion refinement, retrieval call, or human review when the score is below a calibrated threshold. Independently audit a random subset of skipped examples using the expensive ground-truth procedure, and select the largest skip threshold whose exact confidence bound keeps the violation rate below a target budget.

Useful8/10
Difficulty4/10
Novelty7/10
Paper: Audited Selective Verification for Risk-Controlled N-1 Thermal Contingency Screening under Deployment Shift arXiv:2607.13221
△ Mechanism confirmed, baseline not beaten 2026

Joint Tucker-rank and residual-bit KV allocator

Compress each layer's KV tensor with a partial Tucker approximation over token and feature axes, then encode the truncation residual with a rotated uniform quantizer. Select token rank, feature rank, and residual bit-width jointly under a global byte budget, allowing values with flat spectra to receive residual bits while keys may receive more low-rank capacity.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: A JoLT for the KV cache: Near-lossless KV cache compression via joint Lagrangian allocation of Tucker ranks and a rotated residual for llms arXiv:2607.12550
✗ Failed on benchmark 2026

Streaming Contraction Deployment Gate

Attach a streaming contraction monitor to a recurrent, state-space, or neural-ODE model and permit long-horizon rollout or autonomous deployment only when a conservative estimated contraction certificate is positive. The monitor estimates local Jacobian growth from recent state-transition observations and subtracts an uncertainty radius, preventing operation in regimes where apparent stability is caused by insufficient or noisy data.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: Streaming Contraction Certificates for Nonlinear Networks: Topology-Aware Data Sufficiency with Partial Observation arXiv:2607.10893
✗ Failed on benchmark 2026

Fitted CFG terminal step

Replace the standard CFG guidance multiplier in each DDIM step by a coefficient obtained from the terminal guided exponent. Given unconditional and conditional denoiser-derived states D_u and D_c, use r^{1+w}-r instead of w(r-1) on the guidance direction D_u-D_c; this preserves the same two denoiser calls and costs no additional NFE while suppressing low-noise residual blow-up at high guidance.

Useful8/10
Difficulty3/10
Novelty8/10
Paper: Guidance Breaks the Fitted Operator: A Terminal-Fitted Repair for Classifier-Free Guidance arXiv:2607.07665
△ Mechanism confirmed, baseline not beaten 2026

Physics-informed neural decoder

Attach an online PDE-constrained reconstruction layer to a neural operator instead of accepting its raw output field. The layer stays close to the network prediction at sampled output locations while minimizing a differentiable PDE residual and boundary-condition violation, allowing physics correction for each new input without retraining the neural operator.

Useful8/10
Difficulty5/10
Novelty6/10
Paper: Kernel-based Operator Learning: Error Analysis, Budget Allocation, and a Physics-Informed Extension arXiv:2607.06287
✗ Failed on benchmark 2026

Balanced State-Order Compression

Compress each hidden layer by retaining directions that are simultaneously reachable from the observed input distribution and observable at the network output. Unlike PCA or SVD, the retained subspace is weighted by downstream task sensitivity, so high-variance but output-irrelevant directions can be removed while low-variance predictive directions are preserved.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Empirical Minimal-Realisation Compression of Deep Neural Networks via Controllability-Observability Tests arXiv:2607.05457
Audited (legacy) 2026

Asymptotic-preserving terminal completion

Replace the final sequence of diffusion-sampler steps below a positive switching noise scale a with a single analytic normal-mode completion map. Run the existing solver only on [a, sigma_max], then use the denoiser at scale a to extrapolate to the requested terminal floor epsilon. This prevents the step count from growing like log(sigma_max/epsilon) and should preserve the base solver's order when a is coupled to the discretization size.

Useful8/10
Difficulty4/10
Novelty6/10
Paper: Asymptotic Preservation and Uniform Accuracy of Diffusion and Flow-Matching Samplers arXiv:2607.04113
✗ Mechanism failed 2026

Floor-Aware Adaptive Block Drafting

Estimate the irreducible rejection caused by missing within-block information, then use it to choose the block's conditioning order instead of tuning block length blindly. If the estimated floor is high, expose one or more realized tokens before continuing; if the floor is low but observed rejection is high, spend compute on improving the drafter.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Beyond Parallel Blindness: Information Floors and Model Gaps in Block Drafting arXiv:2608.27339
✗ Mechanism failed 2026

Retry-aware ignition-threshold router

Route requests between model-quality tiers using retry-adjusted satisfied-answer throughput instead of nominal completion throughput. Add hysteresis so degradation begins only above an upper backlog threshold and ends only after the backlog is safely below a lower threshold with negative retry-adjusted drift.

Useful8/10
Difficulty4/10
Novelty7/10
Paper: The Shadow Price of Intelligence: Quality Degradation in LLM Inference as a Supply Chain Problem arXiv:2608.23986
✓ Mechanism works 2026

Leave-One-Out Corrective Parallel Sampler

Replace standard tau-leaping in discrete diffusion generation with a first-order sampler whose per-coordinate transition is conditioned on all other current coordinates and excludes the coordinate being updated. After a parallel proposal, use the same leave-one-out conditionals to correct coordinates whose newly sampled values are inconsistent with the rest of the state, allowing large timesteps without permanently propagating simultaneous denoising errors.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Provably adaptive sampling with uniform and remasking discrete diffusion models arXiv:2608.23554
✗ Failed on benchmark 2026

Derivative-Jet MLP Compression

Compress a trained wide analytic-activation MLP by fitting a narrow same-depth student to the teacher's function values and input derivatives, rather than matching only outputs on a calibration dataset. Choose the student width from the input dimension and target error, with a target scaling m = O((log(1/epsilon))^d_in), and use sequential layer fitting plus channel reweighting to limit error accumulation through depth.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Width-Independent Compressibility of Deep Neural Networks arXiv:2608.21752
△ Mechanism confirmed, baseline not beaten 2026

Utility-Weighted Left-Edge Quantization

Replace MSE-calibrated scalar quantization with a conservative left-edge quantizer whose thresholds are denser where activation probability and task utility slope are both high. For a monotone utility function, this should preserve high-impact activation regions better than uniform or MSE-optimal bins at the same number of codes, while retaining an explicit rate-versus-quality design rule.

Useful7/10
Difficulty4/10
Novelty7/10
Paper: A Quantization Problem Posed by Adaptive Streaming arXiv:2609.03745
✗ Failed on benchmark 2026

Certified Coarse-to-Fine Coordinate Refinement

Use the paper's certified-well geometry to turn continuous localization into a cheap grid proposal stage followed by fixed-step refinement. Threshold the projection-residual score on a coarse grid, then run a bandwidth-calibrated gradient map only from accepted points and merge converged duplicates. This avoids dense optimization from every possible coordinate and is suitable for neural slot or source heads that must return a variable number of continuous locations.

Useful7/10
Difficulty4/10
Novelty8/10
Paper: Certified Spherical MUSIC for 3D Localization under Adversarial Subspace Perturbations arXiv:2609.03264
✗ Mechanism failed 2026

Recorded-Mesh Neural ODE Backpropagation

Run an adaptive neural ODE solver once to determine accepted step sizes, then train using a regular fixed-length replay of those steps rather than differentiating through adaptive accept/reject logic. The replay can be fused across a batch of trajectories and differentiated with an ordinary reverse sweep, giving the exact discrete gradient of the replayed solver and predictable GPU work.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: GRADSOLVE: fast exact gradients for ODE ensembles on GPUs arXiv:2609.02876
△ Mechanism confirmed, baseline not beaten 2026

Koopman-MPC Trust Region for Neural Rollouts

Use the adapted linear latent model as a cheap receding-horizon planner or training-time controller around a nonlinear neural predictor. Optimize a short sequence of latent corrections with a quadratic objective, while constraining latent states and inputs to remain inside the region where the Koopman approximation has been identified and its transition spectrum is stable.

Useful7/10
Difficulty7/10
Novelty7/10
Paper: Physics-based Online Adaptive Koopman Model Predictive Attitude Control for Combined Spacecraft with Dynamic Uncertainties arXiv:2609.02534
✗ Mechanism failed 2026

Polar Slack Attention

Use a spherical-design codebook and the paper's polar slack factorization to create a nonnegative geometric interaction bias for attention or expert routing. The resulting kernel is generated by a rank-one term and a rank-at-most-d term, and entries close to zero can define a structured sparse mask instead of relying only on learned top-k selection.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Dual Geometry of Spherical Designs: Polarity, Self-Polar Rigidity, and Quadrature Structure arXiv:2609.02439
✗ Mechanism failed 2026

Ranking-Aware Projection Dimension Rule

Replace the usual JL dimension choice with a task-specific dimension budget based on the paper's asymptotic ranking law. For retrieval, nearest-neighbor search, or compressed attention keys, choose m from a target Kendall correlation and automatically expand the projection when measured ranking quality falls below the target.

Useful7/10
Difficulty3/10
Novelty7/10
Paper: Exact Limits of Random Projections for Preserving Geometry: Distance Recovery, Nearest-Neighbor Rankings, and Covariance Shape in Gaussian Models arXiv:2609.02155
✗ Failed on benchmark 2026

Centered-Geometry Projection Loss

Train a low-dimensional projection of embeddings against centered pairwise geometry instead of only using raw-distance preservation or a JL-style guarantee. The loss removes the population or minibatch distance baseline before comparing distances, forcing the bottleneck to retain the fluctuations that carry ranking and task information.

Useful7/10
Difficulty4/10
Novelty6/10
Paper: Exact Limits of Random Projections for Preserving Geometry: Distance Recovery, Nearest-Neighbor Rankings, and Covariance Shape in Gaussian Models arXiv:2609.02155
✗ Mechanism failed 2026

Riccati-Gated Observation Skipping

Add an uncertainty-aware observation scheduler to a neural state-space model or recurrent world model. Between expensive observation-encoder updates, propagate the latent state using the learned dynamics; periodically compute a decimated Riccati prediction and choose the largest skip length whose predicted covariance remains below a task-specific bound. This replaces a fixed observation stride with a principled, state-dynamics-dependent schedule.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Efficient Sensor Fusion Through Covariance-Constrained Observation Decimation (CCOD) arXiv:2609.02010