In the Shadow of RoBERTA: Is the Classical ML Drawing Its Last Breath in Sentiment Analysis?

Sentiment analysis of textual data is useful in various important tasks. In this paper we use sentiment analysis to evaluate two traditional Machine Learning (ML) models: Stochastic Gradient Descent (SGD) with log loss function and Maximum Entropy (MaxEnt) with two feature selection methods, namely Information Gain (IG) and Pearson correlation (PC) on Czech Facebook Dataset (CFD). We use unigrams as features and demonstrate that F1 score can be significantly improved by using PC as a feature selection method in combination with the SGD model. Next, as a comparison to the current state-of-the-art model RobeCzech (Roberta architecture) we achieved 75.90% F1 score, when no feature space pruning was applied. We interpret this result as an indication that despite the existence of superior RoBerta-based architectures, classical ML models can still be useful in the field of Sentiment Analysis in the Czech and Slovak languages.