Lipschitz-Controlled Metric Projected Optimizer

1. Integration point: modify the optimizer update for a network trained under a convex constraint set S, such as an L2 parameter ball, nonnegative weights, or simplex-valued mixture/router probabilities. Let theta be the parameter vector, g_t the minibatch gradient, and A_t a diagonal SPD preconditioner. For the MVP, derive A_t from Adam's second-moment estimate but explicitly control it rather than using unconstrained Adam scaling. 2. Pseudocode: compute g_t; update v_t=beta2*v_{t-1}+(1-beta2)*g_t^2; set raw_a_t=sqrt(v_t)+epsilon; clip each coordinate to [a,b]; enforce metric variation with a_t=clip(raw_a_t, a_{t-1}(1-rho), a_{t-1}(1+rho)); form z_t=theta_t-eta*(a_t inverse elementwise)*g_t; set theta_{t+1}=argmin_{y in S} 0.5*sum_i((y_i-z_{t,i})^2/a_{t,i}), the weighted projection. For an L2 ball, solve the single Lagrange multiplier by bisection; for a simplex, use weighted simplex projection. 3. Computed from the mathematics: eigenvalue bounds a,b, the variation cap rho, and the weighted projection. Estimated empirically: strong-monotonicity proxy mu_hat=<g_t-g_{t-1},theta_t-theta_{t-1}>/||theta_t-theta_{t-1}||^2, Lipschitz proxy L_hat=||g_t-g_{t-1}||/||theta_t-theta_{t-1}||, and variation ratio ||A_t-A_{t-1}||/||theta_t-theta_{t-1}||. 4. First experiment: train a small MLP on MNIST with weights constrained to an L2 ball, comparing projected SGD, AdamW plus Euclidean projection, clipped adaptive preconditioning, and unclipped adaptive preconditioning. Match learning-rate tuning and optimizer FLOPs. Sweep rho in {0.01,0.05,0.2}. Success is faster loss descent at equal compute, lower final loss, maintained feasibility, and disappearance of persistent oscillations seen in the unconstrained-metric control. Validate the predicted dynamics separately on a strongly convex quadratic where exponential parameter-distance decay can be measured directly.