Drift-Recentered Latent Rank Regularizer

Integrate this into the training loss of a latent diffusion model, neural SDE, or recurrent state-space model. Record hidden states h[t] in R^d at consecutive solver steps or sequence positions. Compute increments delta[t] = h[t+1] - h[t]. For every center t with a complete window of radius w, form the local mean mu[t] = mean(delta[t-w:t+w]) and the centered covariance C[t] = mean over i of ((delta[i] - mu[t]) outer-product (delta[i] - mu[t])). Add epsilon times the identity matrix to C[t] before eigendecomposition. Compute the smallest d-r eigenvalues using a symmetric eigensolver, or use a low-rank randomized eigensolver when d is large. Add alpha times their sum to the task loss; alternatively use the smooth threshold penalty sum_j softplus((lambda[t,j] - tau) / tau), where tau is the desired noise-floor threshold. Average over windows and batch elements. Compare against no regularizer and against raw second-moment regularization C_raw = mean over i of delta[i] outer-product delta[i]. Start with a two-layer GRU or tiny latent diffusion model on noisy sine waves and sequential MNIST, using d=32, target rank r=4 or 8, window radius w=4, and a grid of alpha values. Measure validation likelihood or reconstruction error, effective local rank above tau, and training overhead. Inject a known time-varying deterministic drift into the hidden trajectory. Success means the re-centred method preserves task performance while reducing active stochastic directions, and remains stable under drift whereas the raw second-moment penalty incorrectly interprets drift as extra rank.