Quantization ideas

Low-precision training and inference formulations that keep accuracy within the benchmark's noise floor.

2 ideas found

Category: Quantization Solves: Memory ML: Attention Clear all

✗ Failed on benchmark 2026

Rotated Tucker residual for outlier-resistant KV quantization

Use a low-rank Tucker reconstruction as a structured backbone and quantize only its residual after an orthogonal rotation. The rotation preserves residual energy but redistributes it across coordinates, reducing dynamic-range imbalance and making 2- or 4-bit uniform quantization less damaging than direct quantization of the original KV tensor.

Useful7/10

Difficulty5/10

Novelty6/10

Quantization Memory Accuracy Stability

Paper: A JoLT for the KV cache: Near-lossless KV cache compression via joint Lagrangian allocation of Tucker ranks and a rotated residual for llms arXiv:2607.12550

Unverified 2026

Power-of-Two Latency Attention

Make Laplacian latency attention hardware-friendly by quantizing the exponential decay and row normalization to powers of two. Exponentials and divisions can then be implemented with lookup tables and integer shifts, enabling a low-bit attention kernel for neuromorphic and edge accelerators.

Useful6/10

Difficulty4/10

Novelty7/10

Quantization Speedup Memory Scalability

Paper: Lapis: Laplacian Spiking Attention via First-Spike Timing and Membrane Leakage arXiv:2608.11865