Leveraging convergence behavior to balance conflicting tasks in multi-task learning

Multi-Task Learning is a learning paradigm that uses correlated tasks to improve performance generalization. A common way to learn multiple tasks is through the hard parameter sharing approach, in which a single architecture is used to share the same subset of parameters, creating an inductive bias between them during the training process. Due to its simplicity, potential to improve generalization, and reduce computational cost, it has gained the attention of the scientific and industrial communities. In the literature, the simultaneous learning of multiple tasks is usually performed by a linear combination of loss functions. Nonetheless, tasks gradients often conflict with each other during losses optimization, and it is not trivial to combine them so that all tasks converge toward their optimal solution throughout the training process. To address this problem, the idea of multi-objective optimization was adopted to propose a method that takes into account the temporal behavior of the gradients to create a dynamic bias that adjusts the importance of each task during backpropagation. The result of this method is to give more attention to tasks that are diverging or not being benefited during the last iterations, ensuring that the simultaneous learning is heading to the performance maximization of all tasks. To evaluate the performance of the proposed method in learning conflicting tasks, sensitivity analysis and a series of experiments were performed on a public handwritten digit classification dataset, and on the scene understanding problem in the CityScapes Dataset. Through the performed experiments, the proposed method outperformed state-of-the-art methods in learning conflicting tasks. Unlike the adopted baselines, the proposed method ensures that all tasks reach good generalization performances at the same time it speeds up the learning curves. (AU)