Integer Natural Evolution Strategies

What the math gives to ML

The paper supplies a principled mutation-scale adaptation rule for variables living on the integer lattice rather than in Euclidean space. Its transferable asset is the double-geometric exponential family: the absolute displacement statistic is sufficient, and its centered value gives an exact natural-gradient signal for dispersion. A practical neural-network use is a low-dimensional black-box optimizer for quantization scales, architecture integers, or layerwise perturbation radii, where each coordinate's mutation size must remain integer-native. The safest first test is to replace Gaussian perturbations in a layerwise zero-order optimizer with double-geometric perturbations and compare objective descent at equal function evaluations.