The Optimal Discounting Parameter of the Power Prior under Predictive Log-Loss

What the math gives to ML

The paper provides a closed-form rule for weighting historical observations under predictive log-loss: the optimal power-prior exponent decreases with both historical sample size and distributional mismatch, while increasing with model dimension. This gives neural-network fine-tuning a principled alternative to binary choices between full replay and discarding old data. The transferable asset is the harmonic effective-sample-size law, which converts an estimated KL shift into a bounded amount of useful replay or regularization. The safest implementation is blockwise or head-only at first, because substituting the raw parameter count of a large neural network would make the discount nearly zero.