Investigating Usability Indicators for the Adoption of AI Models in Heuristic Evaluation

Online Learning Management Systems (LMS) have become widely used solutions over the last few years by educational institutions worldwide. Interest in evaluating the quality of these systems has been increasing, and new research to investigate the usability and user experience (UX) of these platforms has increased over the last decade. One of the common evaluation approaches is the heuristic evaluation of the interface based on selected criteria or indicators that describe well-known usability problems. However, this process remains laborious and challenging, requiring considerable effort from evaluators. Adopting automated methods is still uncommon, and approaches based on Artificial Intelligence (AI), for example, are rare. This article presents a study that investigates the potential adoption of usability indicators (Ui) for using artificial intelligence methods as supportive tools for heuristic evaluation of LMS interfaces. In our study, we developed a methodology to investigate some requirements to identify and select a set of Ui to create datasets for AI models to contribute to LMS interface inspection. The methodology allowed us to highlight a set of Ui to be potentially adopted with Machine Learning (ML) to evaluate LMS interfaces. We highlight a set of necessary assumptions to build datasets that can be used with AI models for heuristic evaluation. The methodological approach we propose can be repurposed to study new usability indicators to analyze other complex software contexts.