Multiplicative-Noise Riccati Preconditioner

Integrate the method into the optimizer update for one or more parameter blocks, not into the forward pass. Every K training steps, represent a block by a low-dimensional state x_t in R^m, for example x_t = [normalized parameter coordinates, momentum coordinates], with m between 16 and 64. Model local training dynamics as dx = (A x + B u)dt + sum_h (C^h x + D^h u)dW^h, where u is the proposed parameter update. Compute the minibatch gradient g_t and use it as the unfiltered control proposal. Estimate A and B by ridge regression of x_{t+1}-x_t on [x_t,g_t]. Estimate C^h and D^h from the covariance of regression residuals, using a rank-r eigendecomposition with r between 2 and 8. Choose Q = I, R = lambda I, S = 0, and solve the backward Riccati equation over a short horizon of 10 to 50 optimizer intervals. At the current step form H = R + sum_h D^{hT} P D^h and Theta = B^T P + sum_h D^{hT} P C^h, then apply u = -H^{-1} Theta x, optionally adding a small gradient forcing term -alpha H^{-1} g_t. Stabilize the solve with H <- H + epsilon I and clip the feedback gain norm. The Riccati correction and feedback are computed from the paper's equations; regression, covariance estimation, horizon choice, and clipping are empirical engineering choices. First test on a two-layer MLP trained on CIFAR-10 against AdamW and Shampoo at equal update FLOPs. Use blocks of size 32, refresh the model every 100 steps, and compare loss after a fixed number of examples, maximum stable learning rate, gradient-norm spikes, and final validation accuracy. A positive result is faster loss descent or a larger stable learning rate without degradation in final accuracy.