Cross-Layer Error Compensation and Finite-Sample Feature-Statistics Matching for Extreme Low-Bit Quantization of Large Language Models

What the math gives to ML

This paper contains a directly transferable mechanism for extreme post-training quantization: optimize each quantized layer against the network-level error accumulated before that layer, rather than against an isolated full-precision layer output. The key asset is the exact nonlinear error recursion, which allows the incoming quantization error to be measured with a forward difference instead of approximating every transformer block by a Jacobian. This gives a practical greedy calibration algorithm that can compensate for errors across depth and should be especially valuable for binary or 2-bit weights, where local reconstruction is badly misleading. The most promising first implementation is a drop-in replacement for layer-local PTQ calibration, with the same group-wise codebook and scales but a compensation target computed from the current quantized prefix.