Frequency Coding over Noisy Sampling
arXiv:2608.00539
2026
Architecture
1 ideas extracted · analyzed Aug 31, 2026
What the math gives to ML
The paper provides a concrete spectral representation of a quaternary symmetric noise channel: a Hadamard transform diagonalizes the channel, turning symbol substitutions into independent coordinate contractions whose strengths are powers of \(\delta=1-4p/3\). This suggests a channel-aware neural representation or quantizer that allocates precision according to transform frequency rather than treating all coordinates equally. The most practical transfer is a Walsh-Hadamard bottleneck for categorical distributions, embeddings, or learned activation compression, with coarser quantization on coordinates that the modeled noise strongly attenuates and finer quantization on stable coordinates.
Ideas from this paper
Unverified
2026
Insert a Walsh-Hadamard transform before a quantized categorical or activation bottleneck and assign coordinate-dependent quantization precision using the attenuation spectrum of a quaternary symmetric noise model. Coordinates corresponding to tensor-product frequencies with many nonzero indices are attenuated by higher powers of \(\delta\), so their quantization can be made coarser with little effect on the reconstructed post-noise representation. This creates a structured, fast transform…
Useful5/10
Difficulty4/10
Novelty6/10