Context and Motivation: Software must be improved continuously to meet the users’ expectations. User feedback from online sources allows developers to include the users in the development process when direct communication is impossible. To address the issues raised by feedback, developers must understand which functionalities the users are discussing. Question/Problem: However, manually relating feedback to requirements is too time-consuming. Automatic classification, on the other hand, struggles with the problem that developers and users use different languages when writing feedback and requirements. Principal Ideas/Results: In this paper, we introduce the FeReRe approach for feedback requirements relation. The approach uses a BERT classifier to perform feedback requirements relation on a per-sentence basis. We evaluate the BERT classifier’s performance on multiple datasets and compare it to the performance of the generative LLM GPT4o. BERT achieves an F2 of 0.91 when trained on all available datasets. GPT4o, on the other hand, performs the task poorly, achieving an F2 of only 0.30. Contribution: The paper presents a novel approach for feedback requirements relation along with multiple manually created datasets for training and testing of the presented approach.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

FeReRe: Feedback Requirements Relation Using Large Language Models

  • Michael Anders,
  • Barbara Paech

摘要

Context and Motivation: Software must be improved continuously to meet the users’ expectations. User feedback from online sources allows developers to include the users in the development process when direct communication is impossible. To address the issues raised by feedback, developers must understand which functionalities the users are discussing. Question/Problem: However, manually relating feedback to requirements is too time-consuming. Automatic classification, on the other hand, struggles with the problem that developers and users use different languages when writing feedback and requirements. Principal Ideas/Results: In this paper, we introduce the FeReRe approach for feedback requirements relation. The approach uses a BERT classifier to perform feedback requirements relation on a per-sentence basis. We evaluate the BERT classifier’s performance on multiple datasets and compare it to the performance of the generative LLM GPT4o. BERT achieves an F2 of 0.91 when trained on all available datasets. GPT4o, on the other hand, performs the task poorly, achieving an F2 of only 0.30. Contribution: The paper presents a novel approach for feedback requirements relation along with multiple manually created datasets for training and testing of the presented approach.