Meta-learning Based on Multi-objective Optimization

Meta-learning aims to enable machine learning systems to learn how to learn, which makes it widely applicable in the field of few-shot learning. In common situations, a meta-learner learns meta-knowledge from the training experiences of a set of related tasks and utilizes this meta-knowledge to enable the model to quickly adapt to new tasks. However, the varying impact of different tasks on meta-knowledge, and the potential conflicts that arise from certain tasks, make it difficult to attain an optimal meta-knowledge acquired from multiple tasks. To overcome these limitations, we introduce the concept of multi-objective optimization to the classical meta-learning algorithm, Model-Agnostic Meta-Learning (MAML), aiming to train and obtain the optimal meta-knowledge. Specifically, we employ a gradient-based multi-objective optimization algorithm to efficiently determine a Pareto-optimal set of tasks. Subsequently, the meta-learner effectively utilizes this set of optimal solutions to establish an optimal initial model. Experimental results on the benchmark dataset demonstrate significant performance improvements in the proposed algorithm. Moreover, even with a limited number of tasks, the acquired meta-knowledge approaches the optimal solution and exhibits good generalization capabilities.