Methods of Data Preparation for Multilingual Sentiment Analysis Using Neural Networks
摘要
The research aims to compare various techniques and methods for text data preparation in terms of improving the quality of the classifier. As the preparation of this type of data involves several stages, the author tested various methods for implementing them in each stage. The research objects are reviews, tweets, and statements from 12 datasets. The author determined that different stages of data preparation have different impacts on the algorithm’s quality. For instance, the order in which the tokenization and translation stages occur is unimportant. Similarly, choosing a translation service does not play a big role. On the contrary, selecting embedding and lemmatization methods significantly impacts the outcome. More specifically, using the correct lemmatization method increases classification quality from 1 to 10% and provides the correct embedding models. Based on the comparison of classification quality results from a series of experiments, the author found that selecting the most suitable data preprocessing techniques affects the quality of the outcomes. This implies that incorrectly chosen methods for implementing this phase of constructing a machine learning model can significantly and negatively impact the final classification results.