Workshop#2 N-grams in NLTK and Tokenization in SpaCy (Hour 3–4)

This is a coherent workshop of Chap. 2 using NTLK technology for N-gram generation and statistics. This workshop consists of two parts. Part 1 introduces N-gram language model using NLTK in Python and N-grams class to generate N-gram statistics on any sentence, text objects, whole document, literature to provide a foundation technique for text analysis, parsing and semantic analysis in subsequent workshops. Part 2 introduces spaCy, the second important NLP Python implementation tools not only for teaching and learning (like NLTK) but also widely used for NLP applications including text summarization, information extraction, and Q&A chatbot. It is a critical mass to integrate with Transformer Technology in subsequent workshops.