Large language models are being trained on big data mostly gathered from the Internet, where not all this data are true and correct data. There is a lot of false and misleading data, and the percentage of these false data increases in social media content. This paper investigates the influence of adding noise for Arabic social media comments content on the accuracy and classification results of BERT model which is one of the first and famous LLM models. Our study utilizes a BERT-based classification model trained on a real-world comment dataset to classify the sentiment. The model’s performance has been analyzed and the sensitivity to noise across different noise percentages to the dataset, exploring the impact of noisy data on the performance of the BERT model, including its sensitivity to false or inaccurate data, and analyzing how the effects of noisy data differ between balanced and imbalanced datasets. We employ various evaluation metrics, like accuracy, precision, recall, and \({F}_{1}\) score, to assess the model’s performance. The findings from our research offer insights into the performance and noise sensitivity of BERT in text classification tasks. Our findings demonstrate that BERT is generally resistant to noise at low-medium levels, however on higher noise injection level the model performance affected negatively. Also, we found that imbalanced datasets are particularly sensitive to false positives due to the majority of negative examples, while balanced datasets sustained similar impact on both false positives and false negatives.