# Dataset-adaptive unitary spectral codec

- ID: 1619
- Canonical URL: https://synthcore.org/idea/1619/dataset-adaptive-unitary-spectral-codec
- API JSON: https://synthcore.org/api/idea/1619.json
- API Markdown: https://synthcore.org/api/idea/1619.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2608.00053](https://arxiv.org/abs/2608.00053)
- Category: quantization
- Solves: memory, accuracy, inference-speedup
- ML areas: quantization, cnn, inference-speedup
- Math tags: linear-algebra, tensor-decomposition, geometry, optimization
- Ratings: usefulness 8/10; difficulty 5/10; novelty 7/10

## Idea description

Replace a fixed DCT or Fourier transform in transform coding with a trainable isometric tensor-network transform whose local gates are learned once on a dataset. Retain the k coefficients with largest magnitude and reconstruct with the exact adjoint transform; the transform remains norm-preserving and fast while adapting its coefficient ordering to the data distribution.

## Mathematical statement

The paper defines a unitary transform T acting on an image x, producing coefficients Tx, retaining a coefficient subset of budget k, and reconstructing with T^{\dagger}. Unitarity means T^{\dagger}T=I, so ||Tx||_2=||x||_2 and the inverse is exact before coefficient truncation. T(    heta) is represented by a tensor network of local gates, with trainable parameters     heta constrained to unitary matrix manifolds. The fixed QFT topology includes controlled-phase gates M_{i,j}=\operatorname{diag}(1,1,1,e^{i\phi_{i,j}}), with \phi_{i,j}=2\pi/2^{j-i+1}, and Hadamard gates. Relaxing these gates gives trainable U(2) gates and phase gates D(\boldsymbol{    heta})=\operatorname{diag}(e^{i    heta_0},e^{i    heta_1},e^{i    heta_2},e^{i    heta_3}). The network applies in O(N\log^2N) time for N pixels. The empirical truncated reconstruction objective is L(    heta)=\frac{1}{L}\sum_{\ell=1}^{L}\|x^{(\ell)}-T(    heta)^\dagger P_k(T(    heta)x^{(\ell)})\|_2^2, where P_k keeps the k largest-magnitude coefficients.

## Key formulas

- $$T(\theta)\mathbf{x}\;\text{is the coefficient transform},\qquad \widehat{\mathbf{x}}=T(\theta)^\dagger P_k\bigl(T(\theta)\mathbf{x}\bigr),\qquad T(\theta)^\dagger T(\theta)=I.$$
- $$L(\theta)=\frac{1}{L}\sum_{\ell=1}^{L}\left\|\mathbf{x}^{(\ell)}-T(\theta)^\dagger P_k\left(T(\theta)\mathbf{x}^{(\ell)}\right)\right\|_2^2.$$
- $$M_{i,j}=\operatorname{diag}\bigl(1,1,1,e^{i\phi_{i,j}}\bigr),\qquad \phi_{i,j}=\frac{2\pi}{2^{j-i+1}}.$$
- $$D(\boldsymbol{\theta})=\operatorname{diag}\bigl(e^{i\theta_0},e^{i\theta_1},e^{i\theta_2},e^{i\theta_3}\bigr),\qquad U\in U(2),\quad D\in U(1)^4.$$

## Implementation notes

(1) Integrate the module immediately before entropy coding in a neural image codec, or before a learned scalar or vector quantizer in a compression model. Flatten an HxW image or feature map into N=2^q channels or tokens, apply a fixed FFT/QFT-like butterfly topology, and replace selected 2x2 mixing gates and diagonal phase gates by trainable unitary gates. For real-valued networks, use real orthogonal 2x2 rotations or represent each complex value as two real channels. (2) Pseudocode: initialize T from a normalized DCT or FFT butterfly; for each minibatch x, compute z=T(theta)x; compute mask S=TopK(|z|,k); reconstruct xhat=T(theta)^* (S*z); minimize mean squared distortion plus an optional entropy-model rate estimate; for every gate U, update U <- Exp(skew-Hermitian(A))U, or use a Riemannian optimizer, so U remains unitary. (3) Compute the transform and adjoint by traversing the butterfly tensor network; estimate coefficient entropy empirically with the codec entropy model. The mathematical guarantees are exact norm preservation and invertibility, while gate values and coefficient ordering are learned. (4) First cheap experiment: train on Kodak, CLIC thumbnails, or Quick Draw at 256x256, comparing JPEG/DCT and fixed FFT at identical k or bitrate. Measure rate-distortion, wall-clock transform time, orthogonality error ||T^*T-I||, and gradient stability. Success is lower MSE or bitrate at equal k with less than 2x fixed-transform cost and orthogonality error below 1e-5.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: inconclusive
- Verdict: Built and numerically verified a dataset-adaptive orthogonal top-k spectral codec MVP against an orthonormal DCT baseline. Norm preservation, exact adjoint inversion, and the discarded-energy identity matched to approximately 1e-14; held-out adaptive distortion gains increased strongly with dataset anisotropy, reaching 34.0–54.1% at anisotropy 2.0. The mechanism is real in this toy setting, but fast tensor-network complexity and bitrate benefits were not established.

### Mechanism check

- Verdict: Built and numerically verified a dataset-adaptive orthogonal top-k spectral codec MVP against an orthonormal DCT baseline. Norm preservation, exact adjoint inversion, and the discarded-energy identity matched to approximately 1e-14; held-out adaptive distortion gains increased strongly with dataset anisotropy, reaching 34.0–54.1% at anisotropy 2.0. The mechanism is real in this toy setting, but fast tensor-network complexity and bitrate benefits were not established.
- Confidence: 8/10
- Limitations: Toy N=16 synthetic vectors only; the learned transform is a dense O(N^2) matrix exponential rather than the proposed O(N log^2 N) tensor network. No real images, entropy coding, quantization, bitrate, wall-clock speed, complex gates, or parameter-count comparison was tested.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.12265
- Paired wins: 3/8
- Benchmark verdict: no significant win

## Artifacts

- [REPORT.md](https://synthcore.org/code/480/REPORT.md)
- [bench_report.json](https://synthcore.org/code/480/bench_report.json)
- [codec_experiment.py](https://synthcore.org/code/480/codec_experiment.py)
- [report.md](https://synthcore.org/code/480/report.md)
- [report_bench_2026-08-31T151953.md](https://synthcore.org/code/480/report_bench_2026-08-31T151953.md)
- [results.json](https://synthcore.org/code/480/results.json)
- [spectral_codec_bench.py](https://synthcore.org/code/480/spectral_codec_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/480)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.