JANUS: Online Jacobian-Aligned Infill for Black-Box Optimization

What the math gives to ML

The paper contributes a transferable derivative-free optimization module: estimate local residual geometry from an optimizer's existing evaluation trace, then use that geometry for both exploitation and exploration without replacing the host optimizer. The key asset is the Jacobian-induced metric T=(J^T J+lambda I)^(-1), which aligns search directions with local curvature rather than rank statistics alone. A practical neural-network transfer is black-box optimization of low-dimensional adaptation parameters, quantization scales, or training-schedule variables, where validation feedback can be decomposed into a residual vector and a local Jacobian fitted from recent trials. The strongest first test is to reserve a small fraction of CMA-ES or differential-evolution evaluations for damped Gauss-Newton infill and compare convergence at equal black-box evaluation budgets.