Actionable-Information Optimizer

1. Integration point: replace the raw-gradient input to SGD or Adam with a small controller. At step t construct S_t from the gradient g_t, optimizer second moment v_t, gradient norm, current loss, previous loss, and minibatch gradient variance. Encode S_t with a low-dimensional network E_phi, quantize or perturb it with resolution delta, and obtain Z_t. A policy head outputs a step-size multiplier, direction correction, or accept/reject decision. 2. Pseudocode: compute the base optimizer update a_base; compute z = quantize(E_phi(S_t), delta); enumerate candidate actions a; predict utility u_hat = f_psi(z, a); choose a_star = argmax_a(u_hat - lambda times kappa(z)); apply theta = theta + a_star. After H steps, label the observation with U_t = -(loss_(t+H) - loss_t), and train f_psi to predict this utility. Train the encoder using a variational information penalty or controlled quantization noise. 3. Computed from the mechanism: the finite-resolution observation channel and the acquired-versus-useful information decomposition. Estimated empirically: mutual-information proxies, utility prediction accuracy, update variance, and the largest stable learning rate. Use a variational or classifier-based information estimate initially rather than requiring exact mutual information. 4. First cheap experiment: compare SGD, Adam, and this controller on a two-layer MLP with MNIST and a small ResNet with CIFAR-10. Sweep delta over 8, 4, 2, 1, and 0.5 normalized gradient units, and sweep lambda. The predicted signature is that acquired information increases approximately as -d_eff times log(delta) as delta decreases, while useful information and final loss improvement plateau. Finer resolution beyond the plateau should increase update variance without improving convergence. Sweep learning rates to estimate the divergence boundary; the controller should preserve or increase the maximum stable learning rate relative to raw high-resolution gradients. Require the plateau and boundary shift to replicate across three seeds and two model sizes.