RI-PCGrad: Optimizing multi-task learning with rescaling and impartial projecting conflict gradients

The multi-task learning model is a learning paradigm that shares features across multiple tasks, and it has achieved great success in fields such as computer vision and natural language processing etc. Multiple tasks often conflict and even compete with each other, which seriously reduces the model performance of multi task learning. Most existing optimization methods alleviate multi task gradient conflicts by adjusting task weights. However, it is also important to consider the magnitude and direction of task gradients during the training process. The magnitude and direction of task gradients reflect the conflict and dominance among tasks, which can disrupt the training process and cause instability. In this paper, we present a rescaling and balancing approach for tackling conflicting and dominating gradients. The approach employs a projecting conflict strategy to mitigate the influence of conflicting gradients from multiple tasks and utilize rescaling and balancing techniques to mitigate gradient dominance during training. The proposed method comprehensively considers the weighting, magnitude, and directions of gradients from tasks. We conduct a series of ablation experiments and comparative experiments on different multi-task networks to validate the effectiveness of the proposed algorithm.