Boosting Multitask Decomposition: Directness, Sequentiality, Subsampling, Cross-Gradients
摘要
The exploration of transfer effects and selection of useful auxiliary tasks in multitask learning and foundation models with downstream tasks remain a largely empirical and computationally demanding process. To reduce the computational cost while maintaining statistical rigor, we investigate (1) the concept of direct transfer effect between tasks, (2) the use of sequential learning to minimize the number of test-train data splits, (3) the possibility of using partial data, and (4) the applicability of gradient-based cross-training task affinities in auxiliary task selection. We apply the methods to a drug-target interaction prediction problem.