The analytic hierarchy process (AHP) is a methodology used to derive priority measurements for criteria and sub-criteria, ensuring structured and logical decision-making. However, part of this process asks the subject to make pairwise comparisons between criteria, which can be time-consuming and yield inconsistent results, especially as the subjectivity or number of criteria increases. This experiment introduces an alternative process that presents the subject with all criteria simultaneously, guaranteeing a threshold level of consistency. Subjects were randomly chosen for either the control group, which used the baseline pairwise comparison method, or the experimental group, which used the alternate simultaneous comparison scale method. Each subject was measured for (1) the amount of time it took for them to read the instructions to gain a baseline understanding of the judgment task; (2) the amount of time it took for them to execute the task; (3) the internal consistency rate for their output; and (4) the accuracy of their output compared against an objective standard. The first wave of results (n = 25) found that the alternative method scored with no statistically significant difference to the baseline pairwise comparison method in terms of consistency or accuracy, and while the instructions for the alternate method took longer to read, the task execution was shorter, and thus the total time inclusive of reading and executing was not statistically significant between the two methods. These preliminary results indicate that the alternate method is a viable replacement for the baseline pairwise comparison method; however, it also yielded few advantages. Because the judgments require an objective standard to compare accuracy, a subsequent experiment could focus specifically on the consistency, varying the subjectivity and number of the criteria to determine whether a difference emerges.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Improving Consistency in the Analytic Hierarchy Process: A Comparative Study of Pairwise Comparisons and Simultaneous Comparison Scales

  • Thom Hawkins,
  • Justine Rawal

摘要

The analytic hierarchy process (AHP) is a methodology used to derive priority measurements for criteria and sub-criteria, ensuring structured and logical decision-making. However, part of this process asks the subject to make pairwise comparisons between criteria, which can be time-consuming and yield inconsistent results, especially as the subjectivity or number of criteria increases. This experiment introduces an alternative process that presents the subject with all criteria simultaneously, guaranteeing a threshold level of consistency. Subjects were randomly chosen for either the control group, which used the baseline pairwise comparison method, or the experimental group, which used the alternate simultaneous comparison scale method. Each subject was measured for (1) the amount of time it took for them to read the instructions to gain a baseline understanding of the judgment task; (2) the amount of time it took for them to execute the task; (3) the internal consistency rate for their output; and (4) the accuracy of their output compared against an objective standard. The first wave of results (n = 25) found that the alternative method scored with no statistically significant difference to the baseline pairwise comparison method in terms of consistency or accuracy, and while the instructions for the alternate method took longer to read, the task execution was shorter, and thus the total time inclusive of reading and executing was not statistically significant between the two methods. These preliminary results indicate that the alternate method is a viable replacement for the baseline pairwise comparison method; however, it also yielded few advantages. Because the judgments require an objective standard to compare accuracy, a subsequent experiment could focus specifically on the consistency, varying the subjectivity and number of the criteria to determine whether a difference emerges.