N-Gram Language Model

Language model is the foundation of NLP. This chapter introduces N-gram language model and Markov Chains using classical literature The Adventures of Sherlock Holmes by Sir Conan Doyle (1859–1930) to illustrate how N-gram model works that form NLP basics in text analysis followed by Shannon’s model and text generation with evaluation schemes. This chapter also serves as the conceptual basis for Workshop#2 on N-gram modeling with NLTK in Chap. 11 .