Contraction-Gauge Preconditioning for Quantized Matrix Multiplication
arXiv:2607.18745
2026
Optimization
1 ideas extracted · analyzed Aug 30, 2026
What the math gives to ML
The paper provides a product-preserving reparameterization for quantized matrix multiplication: AB=(AT)(T^{-1}B). Its transferable asset is an exact expected squared product-error formula under independent zero-mean quantization errors, making the choice of representation an explicit optimization problem rather than a heuristic scale search. A practical neural-network adaptation is to select a bounded positive diagonal gauge from calibration data, then quantize the transformed factors. The same objective can be extended to shared gauges across several layers when storage and transformed-copy cost matter.
Ideas from this paper
△ Mechanism confirmed, baseline not beaten
2026
Before quantizing a matrix product, reparameterize its factors as A'=AT and B'=T^{-1}B, preserving the exact full-precision product while changing the quantization difficulty of each factor. Choose a positive diagonal T=diag(t_1,...,t_K) that minimizes predicted post-quantization product error, rather than using output-channel scaling or a fixed heuristic grid. The gauge can be shared across several products when transformed-copy cost matters.
Useful8/10
Difficulty5/10
Novelty6/10