Boosting Sentiment Analysis in OSS: A Hybrid Active Learning Strategy Using Uncertainty Metrics
摘要
Analyzing sentiment in Open-Source Software (OSS) discussions is crucial, but manual labeling for large datasets is prohibitively expensive. We address this annotation bottleneck with a novel hybrid active learning framework. Our approach uses a fine-tuned DistilBERT model and a pool-based strategy that leverages both prediction entropy and breaking-ties margin to efficiently select the most informative comments for annotation. We also include a rigorous, multi-faceted convergence analysis that integrates uncertainty metrics, loss curves, performance trends, and the OracleAcc-MCS stopping criterion. The final model was evaluated on a separate held-out test set, achieving a high accuracy of 0.96 and demonstrating robust generalization. When applied to the full dataset, the converged model revealed a predominant neutral sentiment (66.4%), followed by positive (22.2%) and negative (11.4%) expressions, all with high prediction confidence. This work significantly reduces labeling effort while maintaining high performance, offering a scalable and efficient solution for large-scale sentiment analysis in OSS communication. The fine-tuned DistilBERT model is publicly available to foster further research.