Prompt-Based and Two-Stage Training for Few-Shot Text Classification

Text classification is a crucial task in the field of Natural Language Processing (NLP), which aims at predicting the category where a text belongs. Recently, prompt-based learning has emerged as a powerful approach to handling a wide variety of tasks in NLP. It effectively bridges the gap between pre-trained language models (PLMs) and downstream tasks. Verbalizers are key components in prompt-based tuning. Existing manual prompts heavily rely on domain knowledge, while automatically generated verbalizers, whether on discrete or continuous space, have been suboptimal. In this work, we propose a two-stage training strategy for few-shot text classification, combining prompt-based learning and contrastive learning to learn appropriate verbalizers. In the first stage, we construct positive and negative samples for each input text and obtain soft verbalizers by integrating prompt-based learning and contrastive learning. In the second stage, we leverage the verbalizer learned in the first stage, along with prompt tuning, to train the entire model. Through experiments on some text classification datasets, our method outperforms other existing mainstream methods, which demonstrates its significance.