Learning to learn: a lightweight meta-learning approach with indispensable connections

Meta-learning algorithms learn from other learning algorithms to solve new tasks with only a few labeled instances. Despite being effective for quick learning, it has some limitations. During meta-training phase, inconsequential connections are frequently seen, which leads to an over-parameterized neural network with unnecessary extra gradient computation and memory overhead. To overcome these limitations, we propose a meta-learning method, Meta-LTH, that utilizes the lottery ticket hypothesis technique to prune the neural network using magnitude pruning for retaining essential connections. The pruning process during meta-training generates indispensable connections that can be utilized effectively to solve the few-shot learning problem. Meta-LTH achieves two goals: (a) a sub-network that can adapt more efficiently to meta-learning test tasks, and (b) learns new low-level features of unseen tasks and combines them with the already learned features during the meta-testing phase. Experimental results demonstrate that the Meta-LTH method outperforms the existing first-order MAML algorithm for three classification datasets. The improvement in the classification accuracy by approximately 2% (20-way 1-shot task setting) for the Omniglot and FC100 datasets indicates the ability of Meta-LTH to quickly learn to learn features that are absent in the meta-training data and combine newly learnt feature with the existing features.