Automating User Task Performance: Introducing Task Experience Score (TES) for Complex Cloud Platforms

For IT companies with complex back-end systems, it is crucial to enhance the usability and user experience by facilitating users to operate tasks smoothly and efficiently. Continuously measuring task performance is an essential aspect of experience design and management. Traditional usability metrics in industry and academia often depend on task testing and user feedback, demanding considerable time and resources. There is a scarcity of methods for rapidly, cost-effectively benchmarking and tracking the task performance of multiple products. This paper introduces an automated task performance scoring method and process entirely based on user behavior data for the Alibaba Cloud Console, which encompasses hundreds of cloud products. Utilizing objective raw data captured from web analytics tools, we formulated specific metrics that contribute to the Task Experience Score (TES). This scoring enables various stakeholders, particularly data-sensitive developers, to quickly grasp the experience levels of different products and tasks. TES can also complement other subjective experience measurement indicators, providing more comprehensive guidance for resource allocation and experience optimization in the design of complex back-end products. A preliminary study applied TES on 53 task flows across 10 products within the Alibaba Cloud Console. The results showed that TES closely corresponded with the results obtained through user surveys. Furthermore, we tracked the changes in TES for 8 task flows after optimizing the user interface. The results indicated that TES could evaluate the effectiveness of our design improvements to a certain extent. This simple and efficient method, initially designed for our online cloud platform, is likely adaptable to various types of user interfaces.