Text-Based Sentiment Detection with Ensemble Learning

This paper investigates the combination of Ensemble Learning methods with Bidirectional Encoder Representations from Transformers (BERT) for Natural Language Processing (NLP) models that detect sentiment (negative, neutral, positive) based on text data from Twitter posts. NLP performance is steadily increasing; however, training a new model or optimizing an existing model requires a certain amount of data and expert knowledge. In comparison, this study selected a simpler approach by combining already available and pretrained BERT-based models from the Huggingface platform with Ensemble Learning methods. Two main approaches are evaluated: The Voting Classifier with Hard Voting, Soft Voting and Highest Probability Voting. The Stacking Classifier with different machine learning models as the final meta classifier: Logistic Regression, Naive Bayes and Gradient Boost. The different approaches and implementations are compared. The result shows higher F1-scores when combining existing BERT-based models and using the stacking classifier with gradient boosting model as the metaclassifier. This shows an alternative approach to improving the prediction performance of sentiment detection on Twitter posts by using multiple models with ensemble learning rather than improving an individual model.