LCA-BERT: A Local and Context Fusion Sentiment Analysis Model Based on BERT

Existing sentiment classification models often use static mask matrices for attention calculation when modeling local contexts, which may overlook or excessively focus on other neighboring tokens and fail to adequately represent contextual information in sentiment analysis models. In this paper, we propose a sentiment analysis model called LCA-BERT that enhances local information and integrates contextual information and attention mechanisms based on BERT. Specifically, we replace the static mask matrix in the self-attention network of the BERT model with a dynamic mask matrix, enabling the model to more effectively capture local information. Furthermore, we introduce the concepts of quasi-attention and deep-global context to mitigate the impact of textual noise while capturing contextual semantic information. Comparative experiments with baseline models demonstrate that our proposed model performs better in aspect-based sentiment analysis tasks.