{
 "artifacts": null,
 "category": "optimization",
 "description": "Use the paper's MBM-GP construction to predict input-dependent big-M constants for ReLU disjunctions during neural-network verification. Exact activation-bound optimization is performed only at a small subset of input points, while a Gaussian-process upper confidence bound supplies conservative bounds elsewhere, reducing verifier preprocessing and potentially tightening the MILP compared with one global worst-case constant.",
 "formulas_latex": [
  "$$r_{ij}(y(d))\\leq\\sum_{i^{\\prime}\\neq i}M_{ii^{\\prime}}(d)\\,w_{i^{\\prime}j}(d),\\quad i\\in\\mathcal{I}_{j},\\;j\\in\\mathcal{J},\\;d\\in\\mathcal{D}.$$",
  "$$\\begin{aligned} M_{ii^{\\prime},k}=\\max_{y_k}\\quad\u0026r_{ij}(y_k)\\\\ \\text{s.t.}\\quad\u0026r_{i^{\\prime}j}(y_k)\\leq 0\\\\ \u0026y^{L}\\leq y_{k}\\leq y^{U},\\end{aligned}\\quad k\\in\\mathcal{K}.$$",
  "$$M_{ii^{\\prime}}(d)=\\mu(d)+\\kappa\\,\\sigma(d),$$",
  "$$\\widehat U(x)=\\mu_U(x)+\\kappa\\sigma_U(x),\\qquad h=\\max(0,a)\\ \\Longleftrightarrow\\ [h=0,\\ a\\leq0]\\vee[h=a,\\ a\\geq0].$$"
 ],
 "id": 2766,
 "implementation": "Integrate this into the preprocessing stage of a MILP-based verifier such as a custom ReLU verification model. For every ReLU whose preactivation bound is currently obtained by an expensive LP, identify a low-dimensional domain coordinate: the scalar verification parameter, a one-dimensional perturbation radius, or a projected input coordinate. First choose support points $x_s$ spanning the verification domain and solve the exact LP at each support to obtain $U_s=\\max a(x)$ under the verifier's input constraints. Fit a GP with an RBF or Matérn kernel to $(x_s,U_s)$. At unsolved points, predict the mean and standard deviation and insert $\\widehat U(x)=\\mu_U(x)+\\kappa\\sigma_U(x)$ into the ReLU big-M constraints. Apply the same procedure to $-a$ for lower bounds. Periodically audit random predicted points with exact LPs; if an exact value exceeds the prediction, add that point to the training set and increase $\\kappa$. Pseudocode: `S=initial_supports(); U=exact_LP_bounds(S); gp=fit_gp(S,U); for x in required_points: M[x]=gp.mean(x)+kappa*gp.std(x); if audit and exact_LP(x)\u003eM[x]: S.append(x); U.append(exact_LP(x)); refit(gp); kappa*=1.1; build_relu_MILP(M)`. The mathematical quantities are the exact pointwise bound and GP upper-confidence envelope; kernel choice, support selection, and calibration are empirical. Test first on MNIST or ACAS-X with a 3- to 6-layer ReLU MLP, comparing global interval bounds, exact LP bounds everywhere, and MBM-GP with 5%, 10%, and 20% support coverage. Success requires lower preprocessing or MILP solve time at identical verified accuracy and zero false-safe certificates on a large held-out audit set.",
 "math_summary": "The paper's multiple-big-M constraint is $r_{ij}(y(d))\\leq\\sum_{i'\\neq i}M_{ii'}(d)w_{i'j}(d)$, where $d$ is a domain point, $r_{ij}$ is the constraint residual for disjunct $i$, $w_{i'j}(d)\\in\\{0,1\\}$ indicates an alternative disjunct, and $M_{ii'}(d)$ is a valid relaxation bound. The exact pointwise bound is computed by $M_{ii',k}=\\max_{y_k}r_{ij}(y_k)$ subject to $r_{i'j}(y_k)\\leq0$ and box bounds $y^L\\leq y_k\\leq y^U$. For a ReLU unit with preactivation $a(x)$ and output $h(x)=\\max(0,a(x))$, use the disjunction $[h=0,\\ a\\leq0]\\vee[h=a,\\ a\\geq0]$ and learn the pointwise upper bound $U(x)=\\max_{x'\\in\\mathcal X_x}a(x')$. Fit a GP to exact values $M(x_s)$ at sampled locations $x_s$, with posterior mean $\\mu(x)$ and standard deviation $\\sigma(x)$, and set the conservative bound $\\widehat M(x)=\\mu(x)+\\kappa\\sigma(x)$. Here $\\kappa\\geq0$ is a confidence factor. Since GP coverage is not automatically a proof, calibrate $\\kappa$ on held-out exact solves and fall back to the analytic or global bound whenever the envelope fails.",
 "math_tags": [
  "optimization",
  "probability",
  "statistics",
  "approximation-theory"
 ],
 "ml_areas": [
  "mlp",
  "optimizer",
  "regularization"
 ],
 "paper": {
  "arxiv_id": "2608.27707",
  "arxiv_url": "https://arxiv.org/abs/2608.27707",
  "summary_what_math_gives_to_ml": "The strongest transferable asset is MBM-GP: replace expensive pointwise optimization of a domain-dependent relaxation bound with a Gaussian-process upper confidence envelope. In neural-network verification, ReLU and piecewise-linear activations are encoded by disjunctions whose big-M constants are activation bounds; these bounds vary over the input domain and can dominate verifier cost. A GP surrogate can learn bounds from a small set of exact bound solves, while an upper-confidence prediction remains conservative when calibrated. The practical first target is certified verification of small ReLU networks, measuring whether surrogate bounds reduce preprocessing and MILP solve time without increasing false certificates.",
  "title": "Solution Methods for Infinite-Dimensional Generalized Disjunctive Programming",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 6,
  "novelty": 7,
  "usefulness": 6
 },
 "solves": [
  "speedup",
  "scalability",
  "stability"
 ],
 "title": "GP Upper Bounds for ReLU Verification",
 "url": "https://synthcore.org/idea/2766/gp-upper-bounds-for-relu-verification",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}