Randomly Permuted Orthogonal Products and Fast Dimension Reduction
arXiv:2608.18557
2026
Architecture
2 ideas extracted · analyzed Sep 1, 2026
What the math gives to ML
The paper gives a constructive way to turn orthogonal transforms into nearly incoherent orthogonal transforms by inserting a uniformly random signed permutation. The resulting product spreads signal energy across coordinates while preserving Euclidean norms exactly, which is useful before coordinate sampling, activation pruning, or low-bit quantization. Its ORA construction further provides a sparse implementation using only pairwise pi/4 rotations, with logarithmic parallel depth. The most promising neural-network transfer is a fixed blockwise orthogonal mixer placed immediately before a lossy compression operation.
Ideas from this paper
✗ Mechanism failed
2026
Precondition activation or cache blocks with a fixed product U = A Sigma B of orthogonal transforms and a random signed permutation before quantization or coordinate sampling. The random permutation makes the product incoherent, so energy is less concentrated in a few coordinates and lossy compression should introduce less worst-case distortion.
Useful7/10
Difficulty5/10
Novelty5/10
Unverified
2026
Construct a sparse fixed orthogonal mixer by repeatedly applying pi/4 rotations to randomly matched pairs of feature coordinates. Place this mixer before top-k feature pruning, sparse projection, or activation quantization so that information is spread across coordinates without using a dense random matrix.
Useful6/10
Difficulty4/10
Novelty6/10