Basics of Natural Language Processing

In this chapter, we review the basics of Natural Language Processing and we apply them to the problem of sentiment analysis from text data. Generally, sentiment analysis is performed based on the processing of natural language, the analysis of text and computational linguistics. Although data can come from different data sources, in this chapter we analyze sentiment in text data, using two particular text data examples: one from film critics, where the text is highly structured and maintains text semantics; and another example coming from social networks, where the text can show a lack of structure and users may use text abbreviations. We review basic mechanisms required to perform sentiment analysis, including data cleaning, producing a general representation of the text, and performing some statistical inference on the text represented to determine positive and negative sentiments.