The research aims to compare various techniques and methods for text data preparation in terms of improving the quality of the classifier. As the preparation of this type of data involves several stages, the author tested various methods for implementing them in each stage. The research objects are reviews, tweets, and statements from 12 datasets. The author determined that different stages of data preparation have different impacts on the algorithm’s quality. For instance, the order in which the tokenization and translation stages occur is unimportant. Similarly, choosing a translation service does not play a big role. On the contrary, selecting embedding and lemmatization methods significantly impacts the outcome. More specifically, using the correct lemmatization method increases classification quality from 1 to 10% and provides the correct embedding models. Based on the comparison of classification quality results from a series of experiments, the author found that selecting the most suitable data preprocessing techniques affects the quality of the outcomes. This implies that incorrectly chosen methods for implementing this phase of constructing a machine learning model can significantly and negatively impact the final classification results.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Methods of Data Preparation for Multilingual Sentiment Analysis Using Neural Networks

  • Roman D. Golovin

摘要

The research aims to compare various techniques and methods for text data preparation in terms of improving the quality of the classifier. As the preparation of this type of data involves several stages, the author tested various methods for implementing them in each stage. The research objects are reviews, tweets, and statements from 12 datasets. The author determined that different stages of data preparation have different impacts on the algorithm’s quality. For instance, the order in which the tokenization and translation stages occur is unimportant. Similarly, choosing a translation service does not play a big role. On the contrary, selecting embedding and lemmatization methods significantly impacts the outcome. More specifically, using the correct lemmatization method increases classification quality from 1 to 10% and provides the correct embedding models. Based on the comparison of classification quality results from a series of experiments, the author found that selecting the most suitable data preprocessing techniques affects the quality of the outcomes. This implies that incorrectly chosen methods for implementing this phase of constructing a machine learning model can significantly and negatively impact the final classification results.