The Code Distortion Problem

What the math gives to ML

The paper defines a basis-independent distortion for a linear map between two code subspaces by comparing the largest and smallest multiplicative changes in Hamming weight. This supplies a concrete objective for representation adapters: instead of merely matching mean activations or reconstruction error, enforce that a learned linear map preserves the relative sparsity of directions in a feature subspace. The most practical transfer is a soft, minibatch-estimated distortion regularizer for sparse autoencoders, feature-transfer adapters, or pruning-aware linear layers. Because the exact worst-case objective is combinatorial for neural activations, the implementation should use differentiable soft support together with random and adversarial direction sampling.