Regularity-Aware Stochastic MGDA with Adaptive Conflict-Avoidant Update Direction Control

What the math gives to ML

The paper identifies a non-smoothness mechanism in stochastic multi-objective optimization: the conflict-avoidant MGDA direction is generally only 1/2-Hölder continuous in the objective-gradient Jacobian, but becomes Lipschitz on regular subproblems. This distinction is directly transferable to multi-task neural networks, where noisy mini-batch gradients can make simplex-based gradient combination unstable. A practical transfer is to use MGDA/CA updates only when the gradient Gram matrix has a stable, well-conditioned active set, and fall back to a fixed scalarization when the subproblem is nearly degenerate. The resulting method is a principled adaptive controller for reducing direction variance rather than another static gradient-balancing heuristic.