Quantifying User Experience Through Self-reporting Questionnaires: A Systematic Analysis of the Sentence Similarity Between the Items of the Measurement Approaches

Standardized questionnaires are a common way to collect self-reported data about users. User experience (UX) questionnaires are metrics with multi-dimensional factors. These factors represent specific UX qualities based on items measuring the user’s subjective perception. However, UX questionnaires vary in factor sets, factors names, or measurement items. This study examined 705 items of such factors from 27 popular UX questionnaires. This study aimed to identify the underlying semantic similarity of UX factors on the measurement item level. Augmented SBERT was applied to measure the sentence similarity between the items. Highly similar items were then grouped based on their cosine similarity to identify similarity clusters by the Fast Clustering approach. As a result, 14 similarity clusters could be identified.