Counting and Indexing Words

So far in this book, we did not use the concept of word. This chapter describes how to segment a text in words and sentences, either using regular expressions or a classifier. It also explains how to count the words of a text and index all the words contained in a collection of texts. Finally, it outlines a document representation using vectors of word frequencies and applies this representation in a new PyTorch model to classify texts.