Assessing Code Review Quality with ChatGPT: A Survey of Automated Reviewer Assignment Methods and Experimental Outcomes

In software development, the efficiency of the code review processes is paramount and necessitates intelligent reviewer recommendations. This article explores various methodologies—including machine learning, heuristic-based algorithms, and social network analysis—to automate the process of reviewer suggestion. We examined the RevFinder, TIE, WhoReview, and RSTrace+ approaches, considering file paths, commit messages, reviewer expertise, workload, and response time. We conducted an experiment to assess the review quality annotation abilities of gpt-3.5-turbo and gpt-4 compared to domain expert annotations. The gpt-4 model produced promising results in automating the evaluation of code review quality, aligning with expert opinions 69% of the time. The gpt-3.5-turbo model failed to exhibit similar capabilities. Future research should focus on optimising the practical application of large language models such as gpt-4, concentrating on the reliable handling of linguistic and technical nuances, and the development of industry-specific private models. These models must ensure secure and effective reviewer recommendation in software development, as well as catering to the unique needs and challenges of the industry.