Projective Compact Prior for Latent Models

Modify the latent regularization term of a VAE or deterministic autoencoder. Let the encoder output \(q_\phi(z\mid x)=\mathcal N(m_\phi(x),\operatorname{diag}(s_\phi^2(x)))\), or use sampled codes directly for an autoencoder. Maintain running estimates of latent mean \(\mu\) and covariance \(\Sigma\) with exponential averaging, then apply a Cholesky solve to compute \(r^2(z)\). Set a compactness radius, for example \(R^2=2(d+2)\), giving \(\gamma_R=1/(d+2)\); tune \(R\) only on validation reconstruction versus prior tradeoffs. Add \(\beta\mathcal L_{\mathrm{prior}}\) to reconstruction or task loss. Since hard infinity creates unstable gradients, use the differentiable barrier \(-\log(\max(1-r^2/R^2,\varepsilon))\) and an outside penalty \(\tau\operatorname{softplus}((r^2-R^2)/\tau)\). Optionally project sampled codes using the displayed radial projection before decoding. Pseudocode: sample \(z\); update running \(\mu,\Sigma\); compute \(r^2\), \(\gamma_R\), and the barrier; penalize or project boundary violations; backpropagate reconstruction plus prior loss. The mathematical quantities are the calibrated deformation, Mahalanobis geometry, and compact q-Gaussian; running statistics, \(\beta\), and smoothing are empirical. First test a 2-layer VAE on MNIST or CIFAR-10 against an isotropic Gaussian KL prior. Compare reconstruction, held-out likelihood, latent-radius histograms, and robustness under corruptions. Success is bounded latent norms, fewer decoder failures from extrapolative latent samples, and equal or better reconstruction at the same latent dimension.