Polarity Prediction in Tourism Cuban Reviews Using Transformer with Estimation of Distribution Algorithms

The tourism sector has benefited from recent research in the area of natural language processing, where digital platforms on the web offer the opportunity for people to express their opinions about the services and places they visit. The texts of the reviews are unstructured data characterized by high dimensionality, variable size, and complex semantic relationships between words, which has led to the development of neural architectures with a larger number of parameters to optimize. The training of deep neural networks has been approached by methods based on partial derivatives of the objective function and presents several theoretical and practical limitations, such as the probability of convergence to local minima. In this paper, a hybrid method based on Distribution Estimation Algorithms is proposed for fine-tuning an mT5-based Transformer for polarity prediction. For this purpose, a new Spanish dataset is proposed for polarity classification compiled from TripAdvisor reviews of Cuba. Different preprocessing variants are applied and compared in the solution and data imbalance is treated by back translation. The proposed method combined with back translation decreases the mean of the absolute error in the mT5-based Transformer for polarity prediction.