Online Learning of Scale Parameters in Score-Driven Filters

What the math gives to ML

The paper treats the step-size or gain of a score-driven recursion as an online decision variable rather than a fixed hyperparameter. Its transferable asset is the identity between one-step predictive-loss gradients and a product of consecutive score signals, together with bounded mirror-descent updates that prevent explosive gains and persistence terms that pull gains toward a reference value. In neural networks, this suggests learning per-layer or per-parameter-group step sizes online from the alignment between the current gradient and the next gradient, while enforcing a hard trust region through a bounded mirror geometry. The most direct test is an optimizer variant that replaces fixed AdamW learning rates with bounded, coordinatewise gains and compares loss descent, gradient spikes, and robustness under distribution shift.