Lower-bound-guided binary latent bottlenecks

Integrate this into a VAE or autoencoder at the latent interface. Let the encoder produce a normalized latent z in R^n, estimate its effective sparsity k, compute binary measurements b=sign(Az+xi), and let the decoder reconstruct from b; A is a fixed Gaussian or sub-Gaussian matrix and xi is optional uniform dither. First normalize z <- z/(||z||_2+1e-8), since the mathematical model cannot recover latent norm from signs. Estimate k on each minibatch using either ||z||_1^2/||z||_2^2 as an effective-sparsity proxy or the average top-k energy; keep the estimator fixed for the first experiment. For each batch compute the theoretical floor r=C(k/m)^((2-q)/(2+q)), with C treated as a tunable calibration constant, and optimize reconstruction loss plus a hinge penalty lambda*max(0,r-L_rec) only when an overconfident target is being enforced. More importantly, compare several m values chosen by the inverted rule m proportional to k*epsilon^(-(2+q)/(2-q)) rather than arbitrary widths. Pseudocode: z=normalize(encoder(x)); k=estimate_effective_sparsity(z); b=sign(A@z+dither); xhat=decoder(b); r=C*(k/m)**((2-q)/(2+q)); loss=reconstruction(xhat,x)+beta*max(0,r-reconstruction(xhat,x)); update encoder and decoder, keeping A fixed. The theorem supplies the exponent; C, dither amplitude, and the empirical k estimator must be fitted on validation data. Start with MNIST or CIFAR-10, a small convolutional autoencoder, Gaussian A, and compare learned dense one-bit bottlenecks against this budget rule at equal stored bits and equal decoder FLOPs. Success is a predictable reconstruction-error curve with slope close to m^(-(2-q)/(2+q)), and either lower memory at matched error or lower error at matched bit budget.