LT-SAA: A Lightweight Transformer with Sentiment-Aware Attention for Fine-Tuned BERT-Based Sentiment Classification

Sentiment analysis of user-generated content on digital platforms like Twitter is essential for understanding public opinion and emotional polarity across diverse situations. However, Tweets are typically short and informal which is challenging for multi-class sentiment classification (positive, neutral, negative). Existing techniques often rely on generic attention mechanisms or implicit sentiment handling, struggle with complex expression, and perform equally well across all sentiment classes. This paper proposes Lightweight Transformer with Sentiment-Aware Attention (LT-SAA), a novel hybrid model that integrates fine-tuned BERT embeddings with a lightweight transformer block and a sentiment-aware attention layer guided by VADER sentiment scores. The model employs selective fine-tuning of the final six transformer layers of BERT to enhance computing performance while preserving contextual understanding.The model’s capacity to concentrate on emotionally significant tokens is improved by the sentiment-aware attention mechanism, which dynamically weights features using normalized VADER scores. Experiments on the SemEval-2017 Task 4 Sub-task A (English Language) dataset, LT-SAA achievesan F1-score of 0.61, outperforming baseline models including fine-tuned BERT (F1: 0.593), Multi-view Ensemble (F1: 0.523), and Interpolated DNN (F1: 0.587). By utilizing oversampling and class weighting strategies, the proposed method demonstrates improved recall performance and better handling of class imbalance. LT-SAA offers a realistic and scalable framework for real-time social media monitoring by integrating symbolic sentiment cues with efficient transformer attention. It could be applied to cross-domain datasets like Amazon reviews and implicit sentiment detection could be added for broader applicability.