Information-Charged Event-Feedback Optimizer

Integrate this as a wrapper around SGD or Adam, leaving the neural-network architecture unchanged. At each minibatch compute the gradient g_t, loss \ell_t, gradient norm, cosine similarity c_t=g_t^ op g_{t-1}/(\|g_t\|\|g_{t-1}\|), and a cheap curvature proxy such as \|g_t-g_{t-1}\|/(\| heta_t- heta_{t-1}\|+\epsilon). Discretize these features into event labels k: normal step, gradient spike, sign reversal, plateau, or high-loss minibatch. Maintain one small diagonal preconditioner A_k for each label and apply heta\leftarrow heta-\eta A_k g_t. Estimate p_t(k) with an exponential-moving histogram and calculate the information charge relative to a uniform prior q. Pseudocode: compute g_t and z_t; assign k_t=event_classifier(z_t); update p_t; compute I_t=\sum_kp_t(k)\log((p_t(k)+\epsilon)/(q(k)+\epsilon)); select A_{k_t}; update parameters; record loss decrease, step norm, and J_t. The paper supplies the event-conditioned feedback mechanism; event bins, A_k, and the estimator are empirical adaptations. First test on a two-layer MLP on MNIST or CIFAR-10, comparing vanilla SGD, fixed per-event preconditioners, and feedback with an information penalty that discourages rare modes. Match mean step norm and mean loss decrease across methods. The predicted signature is a reduction in \operatorname{Var}(J_T)/\mathbb{E}[J_T]^2 at fixed mean progress, with the reduction increasing with cumulative feedback information and disappearing when event labels are randomly shuffled. Sweep the event threshold and identify the stability boundary in learning rate; feedback should not reduce variance beyond its measured information-cost budget.