Singularities in Multi-Objective Optimization and their Crossing during Continuation

What the math gives to ML

The paper provides a constructive way to traverse sets of Pareto-critical solutions even when the usual scalarization weight becomes infinite and ordinary continuation stops. Its transferable asset is the projective treatment of objective weights: represent the stationarity condition homogeneously and switch projective charts rather than allowing a loss-weight ratio to diverge. This suggests a multi-task training procedure that traces task-loss tradeoffs through singular regions, producing a family of network parameters in one continuation run instead of restarting training for many fixed loss weights. The main practical limitation is that solving the full stationarity system is expensive for large networks, so the first test should use a small shared MLP or low-dimensional adapters while measuring whether continuation recovers better tradeoff solutions per optimization run.