A Prompt-Based Transfer Learning System for Text Classification
摘要
With the advent of huge pretrained models, enhancing knowledge transfer has been an active area of research. The classical pretrain-finetune paradigm has been the widespread adopted technique to apply the pretrained models to downstream tasks. Though it is beneficial and requires minimal task-specific training, it does not help data scarce settings. Here, comes the importance of few-shot and zero-shot transfer where only demonstrations are shown to the model. With the release of GPT-3, there is an increased interest towards prompting strategies and few-shot transfer. Inorder to conflate the advantages of both finetuning and pretraining strategies, prompt-based finetuning was introduced. We propose to perform prompt-based finetuning over RoBERTa pretrained model using multiple manually designed prompt templates and multi-label words to enhance the performance on classification tasks. We considered three classification datasets: SST2, QNLI, MRPC and found a noticeable improvement on SST2 and QNLI.