Capturing the Meaning of Written Text

The vast majority of information in our society is available as written text. Therefore, this chapter describes the extraction of knowledge from written text. In deep neural networks (DNN), words, sentences and documents are usually represented by embedding vectors. While simple embedding creation methods can only be used to approximate the meaning of words, recurrent neural networks (RNN) have the potential to capture the meaning of a sentence. The best known RNN, Long Short-Term Memory (LSTM), can be used as a language model. It predicts the next word in a sentence and can thus acquire the syntactic and semantic structure of a text. Among other things, it can be used to translate from one language to another. The BERT model calculates the “correlation” between the embeddings of all words of a text and derives context-sensitive embedding vectors that grasp much finer nuances of meaning. It is pre-trained on a large text dataset in an unsupervised manner and then adapted to specific tasks on a small labeled dataset. These models have now been shown to nearly match or exceed human performance for a wide variety of semantic tasks. The transformer model extends this approach to translation and generation of texts and other sequences. Further sections are devoted to the description of images by text and the explanation of predictions of deep neural networks.