One-Bang Gradient-Noise Preparation

Integrate this at the optimizer or scheduler level for the first t_p updates of training. Choose a bounded control u to represent injected Gaussian gradient noise, dropout probability, or a multiplicative learning-rate/noise scale; the cleanest MVP is g_t=\widehat g_t+\sqrt{u_t}\epsilon_t, with epsilon_t sampled from a standard normal distribution and u_t in [u_min,u_max]. Maintain exponential moving averages of m_2=||g_t||^2 and m_4=||g_t||^4, then estimate T=m_2/d and a_2=(d/(d+2))m_4/m_2^2-1. First fit the local reduced dynamics by finite differences: run short constant-control probes at u_min and u_max, regress Delta(T,a_2) on (T,a_2,u), and estimate F,G,B_T,B_a. Specify a terminal target, such as minimizing a_2 at fixed T or maximizing the subsequent loss-decay rate over the next K updates. Numerically integrate the fitted two-dimensional model under both extreme controls, use the better endpoint, and implement a single-switch schedule: u_max for k<k_s and u_min afterward, or the reverse. The paper contributes the affine-control and Pontryagin mechanism; the moment coefficients and reachable interval must be estimated empirically for each network and optimizer. The first cheap experiment should use a two-layer MLP on MNIST or CIFAR-10, comparing Adam or SGD with constant noise, cosine noise annealing, random schedules, and the one-switch controller. Measure gradient kurtosis, loss after the preparation window, and later loss-relaxation rate r_K=[L(t)-L(t+K)]/K. The quantitative prediction is that the best preparation is boundary-valued with one switch, while interior smooth schedules should not improve the terminal objective beyond estimation error; the achieved a_2(t_p) should lie inside the envelope generated by constant u_min and u_max trajectories. Vary the control range and test whether the attainable a_2 interval expands monotonically and whether the best relaxation-rate asymmetry saturates at the predicted reachable boundary.