FeReRe: Feedback Requirements Relation Using Large Language Models
摘要
Context and Motivation: Software must be improved continuously to meet the users’ expectations. User feedback from online sources allows developers to include the users in the development process when direct communication is impossible. To address the issues raised by feedback, developers must understand which functionalities the users are discussing. Question/Problem: However, manually relating feedback to requirements is too time-consuming. Automatic classification, on the other hand, struggles with the problem that developers and users use different languages when writing feedback and requirements. Principal Ideas/Results: In this paper, we introduce the FeReRe approach for feedback requirements relation. The approach uses a BERT classifier to perform feedback requirements relation on a per-sentence basis. We evaluate the BERT classifier’s performance on multiple datasets and compare it to the performance of the generative LLM GPT4o. BERT achieves an F2 of 0.91 when trained on all available datasets. GPT4o, on the other hand, performs the task poorly, achieving an F2 of only 0.30. Contribution: The paper presents a novel approach for feedback requirements relation along with multiple manually created datasets for training and testing of the presented approach.