Recent works in few-shot learning (FSL) have explored the incorporation of supplementary self-supervised auxiliary tasks to facilitate inductive knowledge transfer, yielding promising outcomes. Nevertheless, these approaches only optimize the shared parameters of the FSL model by minimizing a linear combination of two or more task losses, along with manually selecting the combination coefficients. Moreover, due to the unknown and intricate relationships between different tasks, such a simplistic linear combination operation is prone to inducing task conflicts, leading to adverse knowledge transfer. To tackle these challenges, we argue that in few-shot learning (FSL) augmented with auxiliary tasks, the emphasis should be laid on enhancing the performance of the primary FSL task. Specifically, to mitigate the aforementioned task conflicts, we introduce a new Gradient-biAsed Multi-task lEarning (GAME) method, which “makes the primary task primary” by considering both gradient direction and loss magnitude. Extensive experiments demonstrate that the proposed GAME method obtains substantial performance improvements over state-of-the-art methods.