Input convex neural networks as surrogates in mathematical optimisation

What the math gives to ML

The paper’s main transferable asset is not a new activation but the deliberate enforcement of input convexity, which turns a neural surrogate into an object with exploitable global geometry. Nonnegative hidden-to-hidden weights preserve convexity while retaining expressive ReLU piecewise-linear behavior, and the resulting network can be embedded in optimization through linear epigraph constraints rather than binary variables in favorable minimization settings. This suggests using ICNNs as optimization-aware MLPs whenever the target response is known or suspected to be convex, and using their convex structure as an inductive bias rather than merely adding a post hoc convexity penalty. The strongest practical test is a matched-capacity comparison against a ReLU MLP on surrogate prediction and downstream solve time.