Identifying Valid User Stories Using BERT Pre-trained Natural Language Models

Currently, in the software industry, agile methodologies are vastly adopted. These methods use natural language for Requirement Engineering (RE) and are mostly done through the usage of User Stories (UST). This translates into a high cost to elicit and map the software requirements. Pre-trained language models have been shown to be able to verify the validity of requirements in UST format. Aiming to further understand their effectiveness, this paper presents a comparison between different BERT models to validate UST and discusses which model is the most efficient for RE. The main contributions contemplate an enhancement of the available datasets and validation models with up to 98% accuracy. We understand that further analyses should consider which aspect is more important in real-world applications: recall or precision.