Sharp Minimax Regret for Infinite-Memory Logistic Prediction
arXiv:2608.26515
2026
Optimization
2 ideas extracted · analyzed Aug 29, 2026
What the math gives to ML
The paper identifies a lag-resolved complexity spectrum for online logistic prediction with infinitely many past inputs: each lag contributes only on the rounds in which it is available, producing Gamma_T(r) = sum_j log(1 + n_{T,j} r_j^2). This is more informative than treating an infinite-memory filter as a finite-dimensional model, because it separates statistical difficulty from truncation bias and weights old coordinates by their remaining prediction horizon. The transferable ideas are a horizon- and envelope-aware preconditioner for causal convolution parameters and a principled rule for choosing how many lags to retain. These mechanisms are most promising for recurrent filters, long convolutional sequence models, and online or continual learning, where uniform treatment of all lag parameters wastes updates and memory.
Ideas from this paper
✓ Mechanism works
Re-invented
2026
Replace a uniform optimizer on the coefficients of a causal long-memory filter with a lag-aware diagonal Online Newton update. The diagonal curvature and prior scale of lag j are normalized using its number of available prediction rounds and envelope radius, so frequently useful recent lags learn rapidly while weak, old lags receive conservative updates.
Useful7/10
Difficulty4/10
Novelty6/10
Unverified
2026
Choose the retained context length by balancing the statistical complexity of adding lag j against the squared prediction bias from discarding it. Unlike a fixed context window, the rule uses both the remaining-horizon spectrum and the estimated tail energy, allowing a model to expand or shrink its memory online.
Useful6/10
Difficulty5/10
Novelty7/10