Exploring the Automatic Classification of Usage Information in Feedback

Context and motivation: User participation and involvement is important for system success. Communication between developers and users is an important part of participation. This communication influences user satisfaction and therefore system success. However, direct communication between users and developers is often not possible and thus developers need other information sources to understand how the users use the system and what improvements they want. Question/problem: Feedback sources like app stores or online forums provide insights on the users’ view. Due to its size this feedback has to be classified automatically. So far classification has focused on rough classification and the opinions of the users. Detailed usage information is more difficult to classify as the classes are more fine-grained. Principal ideas/results: In this paper, we explore in how far it is possible to mine feedback for the usage information underlying the users opinions and wishes. We analyze multiple classification methods and investigate the transferability across different feedback sources. Additionally, we apply multi-stage classifications to improve classifier performance and we experiment with the granularity of the classes. Overall, BERT performs best in almost all experiments and multi-stage classification does not yield improvement. Improvements are possible with more coarse grained classes. Contribution: To our knowledge, this paper is the first to explore the classification of usage information in explicit feedback. This is a first step towards bundling the usage information for individual functionalities.