Contextual and Semantic Novelty in Text

Novelty, anomaly, or out-of-distribution (OOD) detection has been an active research area for decades, since the 1960s [1, 2], due to its widespread applications in various domains [3, 4], such as financial surveillance, health and medical risk, AI safety, network intrusion detection, etc. Recently, this research has also received increased attention in the Natural Language Processing (NLP) domain. For example, many researchers have studied the problem of text classification [5–9], but, as opposed to classical text classification that works under the closed-world assumption, novelty or OOD detection works under the open-world setting, which allows the class of a test instance to be unknown. However, the existing text classification methods that perform novelty or OOD detection are mainly coarse-grained and topic-based. Given a text document, their goal is to detect whether the text belongs to a known topic or an unknown topic. Some examples include detecting product reviews of unknown product categories [7], news articles of unknown topics [6], and natural language commands of unknown intents/domains [9]. This chapter is based on two papers published in 2021 and 2022 [10, 11]. They introduce a new novelty detection problem in the NLP field—fine-grained semantic novelty detection in texts, which we also call contextual novelty detection. In this chapter, we want to present a learning-based agent for semantic novelty detection in text. The agent does not actively interact with the world. The perceptual operators are Deep Neural Networks (DNN) built on text features created from large language models and external knowledge bases.