Design of LoRA Tunning-Assisted Pretrained LLM Structure for Sentiment Analysis in Online E-Commerce
摘要
Sentiment analysis is crucial for e-commerce operations. It helps enterprises understand consumer feedback and optimize products. Pre-training large language models like Bidirectional Encoder Representations from Transformers (BERT), are current mainstream for sentiment analysis. However, when dealing with scenarios of large-scale data streams like online shopping malls, large parameter count and high computational costs of BERT limit the efficiency. Meanwhile, traditional deep-learning models, while easily deployable, are less accurate for long texts and complex semantics. To bridge the gap, this paper introduces the parameter tuning strategy named low rank adaption (LoRA), and develops an integrated approach by extending standard BERT. It is specifically developed for the various scenes of online E-commerce. Simulations in an online-mall environment and real-dataset tests show BERTLoRA matches BERT’s accuracy on product reviews and notably speeds up training and inference. Compared with traditional models, the accuracy of BERTLoRA has increased by 6.25%. Compared with baseline methods, the proposal has only 0.3% of BERT’s parameters, 32% of its memory occupancy, 40% less training time and 7% less inference time.