Scholarlink

<p>Multi-task learning (MTL) enables a single neural network to solve multiple tasks simultaneously, offering efficiency and improved generalization potential through shared representations. A central challenge in MTL is balancing task-specific losses during training to avoid performance degradation. While uncertainty-based loss weighting (UW) is a popular and competitive approach, we argue that it suffers from several limitations, including overfitting, rigid homoscedastic assumptions, and a lack of theoretical grounding for various loss functions. Therefore, we propose <i>Soft Optimal Uncertainty Weighting</i> (UW-SO), a novel loss weighting method that builds on UW by deriving analytically optimal weights and applying softmax normalization with adaptable temperature parameter, thereby alleviating several of the shortcomings of UW. Through extensive experiments across diverse datasets and architectures, we show that UW-SO achieves superior and robust performance compared to a variety of existing loss weighting methods. Additionally, we provide insights into the effects of temperature selection and propose measures to reduce computational demand.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Investigating Uncertainty Weighting for Multi-Task Learning: Insights and Analytical Alternative

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年12月23日

查看全文
  • Lukas Kirchdorfer,
  • Tobias Sesterhenn,
  • Christian Bartelt,
  • Heiner Stuckenschmidt,
  • Lukas Schott,
  • Jan M. Köhler

摘要

Multi-task learning (MTL) enables a single neural network to solve multiple tasks simultaneously, offering efficiency and improved generalization potential through shared representations. A central challenge in MTL is balancing task-specific losses during training to avoid performance degradation. While uncertainty-based loss weighting (UW) is a popular and competitive approach, we argue that it suffers from several limitations, including overfitting, rigid homoscedastic assumptions, and a lack of theoretical grounding for various loss functions. Therefore, we propose Soft Optimal Uncertainty Weighting (UW-SO), a novel loss weighting method that builds on UW by deriving analytically optimal weights and applying softmax normalization with adaptable temperature parameter, thereby alleviating several of the shortcomings of UW. Through extensive experiments across diverse datasets and architectures, we show that UW-SO achieves superior and robust performance compared to a variety of existing loss weighting methods. Additionally, we provide insights into the effects of temperature selection and propose measures to reduce computational demand.