Reset-Integral Sliding Optimizer

1. Integration point: replace the parameter update in a small-batch optimizer by a continuous-time discretization. Maintain parameters theta, the scalar projected residual g, the integral/sliding variable s, and one scalar reset state z. Choose v as a normalized random projection or the gradient direction from the first minibatch; recompute g at every optimizer step. Use a fixed reset period T_r initially, and optionally add event resets whenever s changes sign. 2. Pseudocode: compute g = dot(grad(L_batch, theta), v); update s = s + dt*(lambda*g) + (g-g_previous); update z = z + dt*(-a*z+b*s); if step modulo T_r equals zero or s*s_previous < 0, set z=0; control = -k_s*clip(s/epsilon,-1,1)-k_i*z-eta*g; theta = theta + dt*control; store g_previous = g. Use gradient clipping only as a safety baseline, not inside the primary comparison. 3. Computed quantities are g, s, z, and the update; estimate the disturbance bound empirically as the high quantile of minibatch gradient disagreement, bar_xi = quantile(||grad_batch-grad_reference||). Select k_s above this estimate and sweep through k_s approximately equal to bar_xi. 4. First cheap experiment: train a two-layer MLP on MNIST or CIFAR-10 with SGD, momentum SGD, and Adam baselines at matched effective step sizes. Measure loss, ||s||, z, gradient-noise variance, and divergence frequency for reset periods T_r in {1, 10, 100, 1000}. The falsifiable prediction is a sharp stability transition near k_s=bar_xi: below it, ||s|| and loss variance grow or fail to contract; above it, the sliding residual decays approximately exponentially until a noise floor. At fixed k_s, the long-run parameter/residual radius should scale linearly with bar_xi/(k_s-bar_xi), while changing T_r should alter transient loss but not the measured reaching rate of s by more than 20 percent if the reset-reachability decoupling transfers.