A sentiment analysis model based on dynamic pre-training and stacked involutions

Sentiment analysis is one of the core tasks in natural language processing, and its main goal is to identify and classify the sentiment tendencies contained in texts. Traditional sentiment analysis methods and shallow models often fail to capture the rich semantic information and contextual relationships in text, while increasing the network depth is prone to problems such as network degradation, which has some limitations in terms of accuracy and performance. Based on this foundation, a sentiment analysis model called BERT-Invos (Bidirectional Encoder Representations from Transformers (BERT)-based stacked involutions) is introduced. The model utilizes the dynamic pre-training language model BERT to encode text, enabling rich semantic features and contextual understanding. In addition, the model employs stacked involutions with varying dimensions to extract features and perceive local information, gradually learning different scales and hierarchical representations of the input text. Furthermore, the proposed method incorporates nonlinear activation functions such as ReLU6 and H-Swish to enhance the model’s expression capability and performance, ultimately delivering classification results. In the experiment, a financial news sentiment dataset was utilized for model validation and comparison against other models. The results revealed that the model achieved an accuracy of 96.34% in sentiment analysis tasks, with precision, recall, and F1 score reaching 96.37%, 96.34%, and 96.34%, respectively. Additionally, the loss value could be minimized to 0.07 with stable convergence, thereby enhancing the accuracy of sentiment classification and reducing loss rates. This improvement facilitates better capturing of local patterns in text and addresses the issue of degradation in deep neural networks. Regarding the deep architecture of the proposed model, future work will focus on further exploring optimization techniques for model compression and deployment.