Augmenting NLP Models with Commonsense Knowledge

This chapter focuses on augmenting NLP models with commonsense knowledge to enhance their performance in natural language understanding and generation tasks. We begin by discussing the importance of commonsense knowledge in NLP and the challenges faced by NLP models in reasoning with commonsense. We explore different types of commonsense knowledge and reasoning tasks, including multiple-choice tasks, open-ended QA, constrained NLG, and commonsense probing of language models. We then introduce the various techniques for augmenting NLP models with commonsense knowledge. We discuss the use of structured knowledge bases, such as ConceptNet, and the incorporation of graph networks for encoding structured knowledge. We also examine the augmentation of NLP models with un/semi-structured knowledge sources, such as text corpora and the use of dense passage retrieval for open-ended QA. Furthermore, we explore differentiable reasoning methods, such as DrFact, for reasoning with semi-structured knowledge. Finally, we discuss the use of neural knowledge models, such as COMET and LLMs, for incorporating commonsense knowledge. We explore the generation of commonsense knowledge graphs using LLMs and knowledge distillation techniques to create smaller, specialized commonsense models. We also examine the use of large language models for extracting relevant commonsense knowledge for reasoning.