No-Regret Bayesian Optimization with Finite-Library Input-Warped Kernels

What the math gives to ML

The paper provides a constructive way to adapt the geometry of expensive black-box search without sacrificing high-probability guarantees: maintain a finite library of warped kernels and permit arbitrary history-dependent kernel selection. The key transferable asset is that fixed-kernel confidence bounds can be lifted to the entire library by a union bound, while information gain under a warp is no larger than that of the base kernel on the warped domain. This is directly useful for neural-network hyperparameter and fine-tuning searches, where learning-rate scales, weight-decay scales, batch sizes, and architectural ratios often have radically different geometries.