Continual Learning of Deep Learning for Indonesian Sentiment Analysis

High-level social media usage makes this social media frequently used as one of the sources for sentiment analysis. Sentiment analysis is a field of study that analyzes people’s opinions or evaluations of entities such as products and services. The Bidirectional Encoder Representation from Transformers (BERT) model is a deep learning architecture that achieves state-of-the-art performance for many natural language processing problems, including sentiment analysis. Several further developments have implemented continual learning on the deep learning model. By applying continual learning, the deep learning model continuously learns based on new data while retaining previously learned knowledge. In this paper, we analyze the performance of the BERT model for continual learning in some domains of Indonesian sentiment analysis. Then it will be compared with two standard deep learning models: fine-tuned embedding with CNN and fine-tuned embedding with LSTM. Our simulation shows the BERT model gives the best accuracy for the transfer of knowledge. However, the fine-tuned embedding with LSTM model is better for retain of knowledge. Moreover, our simulation shows that the order of the source domains affects the performance of BERT for both transfer of knowledge and retain of knowledge.