Convexification of mixed-integer quadratic optimization via decision diagrams
arXiv:2608.22815
2026
Architecture
2 ideas extracted · analyzed Aug 29, 2026
What the math gives to ML
The paper gives a constructive way to optimize binary activation patterns coupled to continuous variables under a quadratic objective, by representing partial assignments with decision-diagram states and merging assignments that induce the same boundary-conditioned quadratic residual. The transferable asset is not generic mixed-integer optimization itself, but the state-compression principle: for tree-structured or inverse-tree-structured interactions, a potentially exponential collection of masks can be represented by a small dynamic program. This suggests exact structured gating or expert-selection layers for neural networks whose redundancy, communication, or activation-cost model is sparse in a graph. The approximate-boundary result further suggests linear-size routing and pruning procedures with an explicit controllable optimization gap.
Ideas from this paper
✗ Mechanism failed
2026
Replace independent top-k expert selection with a decision-diagram router that optimizes a quadratic surrogate over binary expert activations and continuous assignment weights. When expert redundancy or communication costs form a tree, partial routing decisions are merged whenever they have the same separator state, turning exponentially many candidate masks into a dynamic program over a small number of graph cuts.
Useful7/10
Difficulty7/10
Novelty8/10
✓✓ Beats tuned baseline
2026
Use an approximate decision diagram to select a structured subset of neurons, channels, attention heads, or attention edges when their quadratic interactions are sparse or inverse-sparse. Merge states that agree on a local interaction boundary and accept a tunable epsilon loss in the pruning objective, obtaining a representation whose size is linear in model width for fixed accuracy tolerance.
Useful6/10
Difficulty6/10
Novelty7/10