Complete-U Moment Regularizer

1. Integration point: add a spectral-moment regularizer to an encoder, projector, or representation model immediately after the embedding tensor z=f_theta(x) is produced. Start with degree two because it is O(B^2) and directly tests the variance mechanism. For z in R^(Bxd), compute the Gram matrix G=z z^T, remove its diagonal, and define U as the mean squared off-diagonal Gram entry. Add lambda*(U-target)^2 when matching a desired second spectral moment, or lambda*U when the goal is to suppress correlation and spectral concentration. 2. Pseudocode: z=encoder(x); G=z @ z.T; off=(G**2)*(1-eye(B)); U=off.sum()/(B*(B-1)); loss=task_loss + lambda*(U-target)2; loss.backward(); optimizer.step(). The baseline uses m=B/2 random disjoint pairs and V=mean((z[2r]^T z[2r+1])2). 3. Compute the paper's predicted variance terms empirically: estimate Var(h) from all off-diagonal pair values, estimate zeta_1 using the variance across i of row means mean_{j!=i} h_ij, and set zeta_2_hat=Var(h)_hat-2*zeta_1_hat. Compare the predicted complete-versus-batched variance with repeated minibatch measurements. 4. First experiment: train a small CIFAR-10 ResNet-18 or MNIST two-layer MLP with identical optimizer, batch size, and regularization coefficient under complete pairs and disjoint pairs. Log regularizer variance, parameter-gradient variance, loss curves, validation accuracy, and wall-clock time. Success is at least 20% lower gradient variance or smoother descent at equal batch size, with no more than 2x overhead and no accuracy loss.