This study evaluates the performance of three predefined sentiment analysis tools in Python—TextBlob, VADER, and transformer-based models—on a dataset of 396 climate change-related tweets. Each tool was combined with classification algorithms (SVM, Naive Bayes, KNN, and Random Forest) to assess emotional responses based on post reach (followers, likes, retweets, and account verification). A subset of tweets was manually annotated to serve as ground truth. Results show that transformer models paired with Naive Bayes achieved the highest F1-Score, while VADER with SVM performed best in terms of ROC AUC. To formally assess tool reliability, Cohen’s Kappa coefficient was used to measure agreement with human labels, revealing limited alignment across tools. These findings highlight the challenge of interpreting sentiment in social media contexts and the need for models attuned to nuance, sarcasm, and emotional ambiguity in environmental discourse.