Purpose <p>Researchers have argued that EQ-5D may not capture all aspects in health-related quality of life (HRQoL), they therefore proposed adding additional dimensions. However, whether additional dimensions are more important than EQ-5D dimensions is unknown. This study aimed to explore the impact of four different ranking tasks on eliciting health preferences and assess which ranking task provides better consistency and test–retest reliability.</p> Methods <p>Besides the five EQ-5D dimensions, nine HRQoL dimensions were selected, including culturally universal dimensions and culturally specific dimensions relevant to Chinese. To evaluate the impact of dimension descriptions and instructions, we tested four ranking tasks derived from two dimension description versions and two instruction versions. Each participant completed the questionnaires twice, enabling intra-group comparisons, including test–retest reliability when using same questionnaire and consistency when using different questionnaires.</p> Results <p>The phrase “moderate problems with [dimension]” yielded more stable rankings than simply using “problems”, even when the instruction varied. When dimension descriptions differed, the correlation between rankings was lower (<i>rₛ</i> = 0.859) than when instructions varied (<i>rₛ</i> = 0.961). Additionally, when using the same instruction, aggregate test–retest reliability of “moderate problems” was higher (<i>rₛ</i> = 0.972) than that of “problems” (<i>rₛ</i> = 0.871). The average median of correlation coefficient of “moderate problems” (0.81 (0.56~0.89)) was higher than that of “problems” (0.70 (0.45~0.79)).</p> Conclusion <p>This study shows that ranking method is feasible and that the dimension description influences the outcome. When compared to “problems” description, the results using “moderate problems” description shows higher aggregate stability and individual-level test–retest consistency.</p>