A systematic literature review of the application of user experience studies in cyberinfrastructure for scientific research

Good usability and user-friendliness of cyberinfrastructure tools and technologies are crucial for increasing their usage and broader adoption in scientific communities. However, there is limited empirical evidence on effectively doing this in practice. This systematic literature review aims to uncover, analyze, and present how user experience (UX) evaluations have been applied in cyberinfrastructure for scientific research. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) framework is applied to analyze 15 articles on UX studies in cyberinfrastructure, published between 2015 and 2022. Results show that cyberinfrastructure use multiple methods for UX evaluation: interviews, think-aloud, and user surveys. However, the UX studies on cyberinfrastructure primarily focus on the technical effectiveness of the system. This review identified limitations in the current evaluation methodologies, including a need for more agreement on what constitutes a UX evaluation process for cyberinfrastructure. Implications are that UX studies of cyberinfrastructure should report clear evidence of user feedback collected during the evaluations. This study highlights the necessity of addressing sociotechnical issues in cyberinfrastructure design by including the voice of all representative stakeholders and keeping the social context in loop during the system development process.