Bounded Telegraph Exploration for Optimizers

Integrate the mechanism into a momentum-like optimizer state rather than directly perturbing weights. For parameter vector theta, maintain telegraph force u and update theta <- theta - eta [g(theta) + alpha u], where g is the minibatch gradient, eta is the learning rate, and alpha scales exploration. Use one scalar telegraph process per parameter block or layer to avoid storing a sign for every parameter. With step size dt, flip each sign s_k independently with probability 1 - exp(-lambda_k dt), then apply the exact linear relaxation update u <- exp(-gamma dt) u + [1 - exp(-gamma dt)] sum_k v_k s_k/gamma. Pseudocode: initialize u = 0 and random signs; for each minibatch compute g; flip signs; update u; optionally clamp only roundoff excursions to [-u_max,u_max]; apply theta <- theta - eta(g + alpha u). The paper supplies the bounded-support and stationary-moment mechanism; estimate empirical variance and autocorrelation from the run. First test a small MLP and ResNet-18 on CIFAR-10 against SGD and SGD with Gaussian OU noise at matched variance. Sweep gamma, K, and lambda_k. The quantitative prediction is |u| <= u_max at every step, with violations below numerical precision. Measured stationary variance should match the closed form within 10%, and the autocorrelation should fit the predicted sum of exponentials. As K increases with fixed total variance, standardized excess kurtosis should decrease toward zero.