Knowledge Prompting with Contrastive Learning for Unsupervised CommonsenseQA

Unsupervised commonsense question answering is an emerging task in NLP domain. In this task knowledge is of vital importance. Most existing works focus on stacking large-scale models or extracting knowledge from external sources. However, these methods suffer from either the unstable quality of knowledge or the deficiency in the model’s flexibility. In this paper, we propose a Knowledge Prompting with Contrastive Learning (KPCL) model to address these problems. Specifically, we first consider dropout noise as augmentation for commonsense questions. Then we apply unsupervised contrastive learning in further pre-training to capture the nuances among questions, and thus help the subsequent knowledge generation. After that, we utilize generic prompts to generate question-related knowledge descriptions in a zero-shot manner, facilitating easier transfer to new domains. Moreover, we concatenate knowledge descriptions with the commonsense question, forming integrated question statements. Finally, we reason over them to score the confidence and make predictions. Experimental results on three benchmark datasets demonstrate the effectiveness and robustness of our proposed KPCL, which outperforms baseline methods consistently.