Position-only active-noise optimizer

1. Integration point: begin with a one-dimensional and then low-dimensional quadratic-loss problem, followed by a two-layer MLP on MNIST. Treat the parameter error e_t = theta_t - lambda_t as the observed position. Maintain a hidden-activity estimate a_hat and covariance P for each parameter block or for a low-rank gradient subspace. 2. Pseudocode: initialize a_hat = 0 and P. At each minibatch, compute gradient g_t and predict a_hat <- exp(-Delta/tau) a_hat; propagate P using the OU process covariance. Predict the gradient residual r_t from the local curvature model, observe r_t from the minibatch gradient, and perform a Kalman measurement update to obtain a_hat. Form u_t = v_0 - G e_t - G_a a_hat, update lambda <- lambda + Delta u_t, and update parameters with theta <- theta - eta (g_t - c a_hat), where c is a tunable cancellation gain. Estimate tau from the empirical autocorrelation of minibatch-gradient residuals, Sigma_a from residual covariance, and K from diagonal Hessian estimates or Hessian-vector products. Compute the linearized augmented closed-loop matrix periodically; reduce eta or gains whenever rho(A_cl) exceeds 0.98. 3. Taken from the paper are the OU latent-disturbance model, inference from position-only observations, and feedback objective; curvature, noise covariance, observation covariance, and gains must be estimated or tuned in the neural-network experiment. 4. First cheap experiment: compare SGD, Adam, momentum, and this controller on a quadratic objective with synthetic OU gradient noise, then on an MLP for MNIST. Sweep tau over 0.1, 1, 10, and 100 optimizer steps. Prediction: filtering gives little benefit for tau much smaller than one step, strongest improvement when tau matches the optimizer relaxation time, and a residual benefit for tau much larger than the training horizon. On the quadratic system, divergence should occur at rho(A_cl) = 1, with the measured learning-rate boundary within 20 percent of the spectral prediction.