# Rotated Tucker residual for outlier-resistant KV quantization

- ID: 814
- Canonical URL: https://synthcore.org/idea/814/rotated-tucker-residual-for-outlier-resistant-kv-quantizatio
- API JSON: https://synthcore.org/api/idea/814.json
- API Markdown: https://synthcore.org/api/idea/814.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.12550](https://arxiv.org/abs/2607.12550)
- Category: quantization
- Solves: memory, accuracy, stability
- ML areas: transformer, kv-cache, attention, quantization
- Math tags: linear-algebra, tensor-decomposition, quantization, random-matrix
- Ratings: usefulness 7/10; difficulty 5/10; novelty 6/10

## Idea description

Use a low-rank Tucker reconstruction as a structured backbone and quantize only its residual after an orthogonal rotation. The rotation preserves residual energy but redistributes it across coordinates, reducing dynamic-range imbalance and making 2- or 4-bit uniform quantization less damaging than direct quantization of the original KV tensor.

## Mathematical statement

Let X be a layer's token-feature cache slice,     ilde{X}(r_T,r_d) its partial Tucker reconstruction, and R=X-    ilde{X} the discarded energy. Let Q be an orthogonal matrix or fast orthogonal transform satisfying Q^TQ=I, so Frobenius energy is preserved: ||QR||_F=||R||_F. For b in {2,4,8}, define Y=QR and quantize each block with symmetric uniform quantization Q_b(Y)=s clip(round(Y/s), -2^{b-1}, 2^{b-1}-1), where s=max_i|Y_i|/(2^{b-1}-1) is the block scale. Decode with \hat{X}=    ilde{X}+Q^TQ_b(QR). The rotation cannot reduce the residual's Frobenius norm; its intended benefit is reducing coordinate outliers and block dynamic range, which lowers uniform-quantization error. For b=0, no residual is stored and \hat{X}=    ilde{X}.

## Key formulas

- $$R=X-\tilde{X}(r_T,r_d),\qquad Y=QR,\qquad Q^{\mathsf T}Q=I,$$
- $$Q_b(Y)=s\,\operatorname{clip}\!\left(\operatorname{round}(Y/s),-2^{b-1},2^{b-1}-1\right),\qquad s=\frac{\max_i|Y_i|}{2^{b-1}-1},$$
- $$\widehat{X}=\tilde{X}(r_T,r_d)+Q^{\mathsf T}Q_b(QR),\qquad \|QR\|_F=\|R\|_F.$$
- $$\operatorname{MSE}_{\mathrm{KV}}=\frac{\|X-\widehat{X}\|_F^2}{\|X\|_F^2},\qquad b\in\{0,2,4,8\},\quad b=0:\ \widehat{X}=\tilde{X}.$$

## Implementation notes

Integrate this into the KV-cache write path for every layer and separately for K and V. Maintain Tucker factors for the selected backbone, compute the reconstructed block X_tilde, and form R=X-X_tilde for each incoming token block. Use a fixed fast orthogonal transform on the residual feature dimension: a normalized Hadamard transform is suitable when the feature width is a power of two, while a randomized orthogonal matrix can be used for an offline prototype. Partition the rotated residual into blocks of 64 or 128 values. For each block, compute s=max(abs(Y))/(2^(b-1)-1), store the scale s and integer codes clip(round(Y/s), -2^(b-1), 2^(b-1)-1). Decode by multiplying codes by s, applying the inverse orthogonal transform, and adding X_tilde. For b=0, skip residual storage and return only the Tucker reconstruction. Log block maxima, scale variance, residual MSE, and attention-output error before and after rotation; norm preservation is guaranteed mathematically, whereas quantization improvement must be measured. Start with a small one-layer test and then full Mistral-7B inference at equal residual bit-width and equal total bytes. Compare direct residual quantization, rotated residual quantization, direct 4-bit KV, and FP16 KV on perplexity, long-context retrieval, decode throughput, transform overhead, and peak memory. Success is lower cache reconstruction error and smaller perplexity degradation at 2- or 4-bit residual precision without making the transform bandwidth dominate attention.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: inconclusive
- Verdict: Built and numerically verified a rotated Tucker-residual quantization MVP. Frobenius norm preservation and the predicted Hadamard outlier-spreading law matched numerical results, while the toy benchmark showed a substantial 4-bit improvement but a 2-bit regression. The mechanism is real but conditional, so this does not establish a universal low-bit KV-cache win.

### Mechanism check

- Verdict: Built and numerically verified a rotated Tucker-residual quantization MVP. Frobenius norm preservation and the predicted Hadamard outlier-spreading law matched numerical results, while the toy benchmark showed a substantial 4-bit improvement but a 2-bit regression. The mechanism is real but conditional, so this does not establish a universal low-bit KV-cache win.
- Confidence: 8/10
- Limitations: This is a NumPy toy experiment with an SVD matrix low-rank backbone rather than full Tucker factors or a transformer KV cache. It does not test perplexity, attention-output error, throughput, memory, transform overhead, varying block sizes, randomized rotations, or end-to-end inference.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.30895
- Paired wins: 4/8
- Benchmark verdict: no significant win

## Artifacts

- [REPORT.md](https://synthcore.org/code/331/REPORT.md)
- [bench_report.json](https://synthcore.org/code/331/bench_report.json)
- [report.md](https://synthcore.org/code/331/report.md)
- [report_bench_2026-08-31T012811.md](https://synthcore.org/code/331/report_bench_2026-08-31T012811.md)
- [results.json](https://synthcore.org/code/331/results.json)
- [rotated_tucker_bench.py](https://synthcore.org/code/331/rotated_tucker_bench.py)
- [rotated_tucker_experiment.py](https://synthcore.org/code/331/rotated_tucker_experiment.py)
- [Download all files as ZIP](https://synthcore.org/download/331)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.