# Projected Regularized Multi-Gradient Optimizer

- ID: 3079
- Canonical URL: https://synthcore.org/idea/3079/projected-regularized-multi-gradient-optimizer
- API JSON: https://synthcore.org/api/idea/3079.json
- API Markdown: https://synthcore.org/api/idea/3079.md
- Verification status: unverified
- Source: [arXiv:2609.02722](https://arxiv.org/abs/2609.02722)
- Category: optimization
- Solves: stability, accuracy, sample-efficiency
- ML areas: optimizer, training, fine-tuning
- Math tags: optimization, convex-analysis, probability
- Ratings: usefulness 6/10; difficulty 5/10; novelty 5/10

## Idea description

Replace a scalarized multi-task loss with an adaptive projected update that chooses a regularized convex combination of task gradients at every step. The combination is conservative when task gradients conflict, while the regularization is adapted from the actual projected step length rather than from loss values or a line search.

## Mathematical statement

The paper formulates the population stochastic multi-objective problem as $\min_{\mathbf z\in\mathcal X}F(\mathbf z)$ with $F(\mathbf z)=(F_1(\mathbf z),\ldots,F_m(\mathbf z))^\top$ and $F_i(\mathbf z)=\mathbb E[f_i(\mathbf z,\boldsymbol\xi)]$, where $\mathcal X\subseteq\mathbb R^n$ is nonempty, closed, and convex, $m$ is the number of objectives, and $\boldsymbol\xi$ is a random sample. For a minibatch, let $g_i=\nabla_\theta \widehat F_i(\theta)$ be the stochastic gradient of task $i$, let $\Delta_m=\{\alpha\in\mathbb R^m:\alpha_i\ge 0,\sum_i\alpha_i=1\}$ be the probability simplex, and let $G=[g_1,\ldots,g_m]$. Use the regularized multi-gradient $d=G\alpha$ with $\alpha$ solving $\min_{\alpha\in\Delta_m}\frac12\|G\alpha\|_2^2+\frac{\rho}{2}\|\alpha\|_2^2$, where $\rho>0$ prevents unstable or nearly non-unique task weights. The feasible update is $\theta^+=P_{\mathcal X}(\theta-\eta d)$, where $P_{\mathcal X}$ is Euclidean projection and $\eta>0$ is the base step size. Define the projected displacement $r=\|\theta^+-\theta\|_2/\eta$ and adapt $\rho$ using an exponential moving-average target $s$. The constrained Pareto-stationarity certificate is the projected residual $\|\theta-P_{\mathcal X}(\theta-d)\|_2$, which is zero when the selected common direction is compatible with the normal cone of $\mathcal X$.

## Key formulas

- $$\min_{\mathbf{z}\in\mathcal{X}}\;F(\mathbf{z}):=\big(F_{1}(\mathbf{z}),\ldots,F_{m}(\mathbf{z})\big)^{\top},\qquad F_i(\mathbf z):=\mathbb E[f_i(\mathbf z,\boldsymbol\xi)].$$
- $$\alpha_k\in\arg\min_{\alpha\in\Delta_m}\left\{\frac12\left\|G_k\alpha\right\|_2^2+\frac{\rho_k}{2}\|\alpha\|_2^2\right\},\qquad d_k=G_k\alpha_k,\quad G_k=[g_{1,k},\ldots,g_{m,k}].$$
- $$\theta_{k+1}=P_{\mathcal X}(\theta_k-\eta_k d_k),\qquad r_k=\frac{\|\theta_{k+1}-\theta_k\|_2}{\eta_k}.$$
- $$\rho_{k+1}=\operatorname{clip}\!\left(\rho_k\left(\frac{r_k+\varepsilon}{s_k+\varepsilon}\right)^\gamma,\rho_{\min},\rho_{\max}\right),\qquad s_{k+1}=\beta s_k+(1-\beta)r_k.$$

## Implementation notes

Integrate this as a replacement for the optimizer update after backpropagation, especially for multi-task learning with losses $\ell_1,\ldots,\ell_m$. First choose the parameter constraint set $\mathcal X$: use a large Euclidean ball around the pretrained weights for fine-tuning, a per-layer norm ball, or simple elementwise bounds for a cheap prototype. For each minibatch, compute one gradient $g_i=\nabla_\theta\ell_i$ per task, flatten or represent them as vectors, and form $G=[g_1,\ldots,g_m]$. Solve the small quadratic program over the simplex using projected gradient or Frank-Wolfe: initialize $\alpha$ uniformly, repeatedly compute $q=G^T G\alpha+\rho\alpha$, take $\alpha\leftarrow\Pi_{\Delta_m}(\alpha-\tau q)$ for 10--30 iterations, and set $d=G\alpha$. Apply $\theta^+=P_{\mathcal X}(\theta-\eta d)$; for a ball projection, rescale only when the norm constraint is violated. Measure $r=\|\theta^+-\theta\|/\eta$, update the EMA target $s\leftarrow\beta s+(1-\beta)r$, then update $\rho$ with the displayed clipped power rule. The mathematical quantities are the simplex QP, Euclidean projection, and projected residual; $\eta$, $\gamma$, $\beta$, and clipping limits must be tuned empirically. Begin with a small ResNet-18 or shared MLP on CIFAR-100 with three tasks or three data augmentations, comparing against AdamW, fixed-weight loss scalarization, and unregularized MGDA at equal backward-pass and wall-clock budgets. Success means fewer task regressions, lower worst-task validation loss, and smaller variance of task losses; also report the extra simplex-QP cost and whether the projected residual decreases more reliably under strong gradient conflict.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.