Mean-Preserving Diversity Regularizer

1. Integration point: use the loss for a conditional VAE, diffusion model with multiple noise draws, mixture-density head, or stochastic graph-neural-network predictor. For each minibatch source x_i, draw K independent noises and produce target samples y_ik; retain the ordinary task likelihood or diffusion denoising loss. 2. Pseudocode: for each x_i, generate y_i1,...,y_iK and set z_i=mean_k(y_ik). Compute source Gram matrix Gx_ii'=dot(x_i,x_i') and barycentric target Gram matrix Gz_ii'=dot(z_i,z_i'); set L_bar=mean_{i,i'}(Gx_ii'-Gz_ii')^2. Draw Q random vectors u_q and biases c_q, define convex phi_q(y)=softplus(u_q^T y+c_q), and compute L_cx=mean_{i,q} relu(mean_k phi_q(y_ik)-phi_q(z_i)). Optimize L_task+lambda_bar*L_bar+lambda_cx*L_cx. Use stop-gradient on z_i in L_cx only if the model inflates variance by moving the mean. 3. The barycentric relation and convex-order condition come from the paper; the finite softplus family is an empirical approximation to the all-convex-functions definition. Monitor mean preservation with mean_k(y_ik)-z_i and tune lambda values on validation likelihood. 4. First cheap experiment: train a conditional VAE on MNIST or CIFAR-10 where each input is paired with several valid augmentations, or use synthetic data y=x+s with zero-mean multimodal s. Compare standard reconstruction/KL training against the added loss using K=8 and Q=16. Evaluate conditional sample diversity, squared error of the generated sample mean, pairwise relation error, and mode coverage. Success is higher diversity and mode coverage at nearly unchanged mean error and task likelihood; failure is variance inflation, detected by calibration and off-support sample metrics.