Proximal Dry-Friction Lookahead Momentum

Integrate the method at the optimizer-update level, not inside the network forward pass. Store a velocity tensor v with the same shape as every parameter tensor x. At step k, create look-ahead parameters y=x+beta*v, run one forward/backward pass at y, and obtain g=grad L(y). Then apply the semi-implicit update independently to each parameter tensor: z=(v-h*g)/(1+h*gamma); lambda=h*r/(1+h*gamma); v=sign(z)*max(norm(z)-lambda,0); x=x+h*v. Here norm(z) is the Euclidean norm of the tensor or parameter group, and the shrinkage factor is broadcast over that group. Implement both per-tensor and per-layer grouping, because a single global norm can let large layers dominate. The proximal formula is computed exactly; no Hessian is estimated. As an optional second variant, divide g by an EMA RMS preconditioner before applying the velocity and proximal update, but report this separately because it changes the underlying metric. Log loss, validation accuracy, velocity norms, the fraction of groups with exactly zero velocity, gradient norms, and the number of update-direction sign reversals. Begin with a two-layer MLP on MNIST and a ResNet-18 on CIFAR-10. Compare with SGD, SGD with momentum, Nesterov momentum, and AdamW using matched gradient evaluations. Sweep beta in {0, 0.05, 0.1, 0.2}, gamma in {0, 0.01, 0.1}, and r as {0.001, 0.01, 0.1} times the median initial gradient norm. Success means lower late-training loss variance and fewer sign reversals at equal accuracy; a stronger result is frequent exact zero velocity with equal or better validation accuracy and no meaningful wall-clock penalty.