Improving Consistency in the Analytic Hierarchy Process: A Comparative Study of Pairwise Comparisons and Simultaneous Comparison Scales
摘要
The analytic hierarchy process (AHP) is a methodology used to derive priority measurements for criteria and sub-criteria, ensuring structured and logical decision-making. However, part of this process asks the subject to make pairwise comparisons between criteria, which can be time-consuming and yield inconsistent results, especially as the subjectivity or number of criteria increases. This experiment introduces an alternative process that presents the subject with all criteria simultaneously, guaranteeing a threshold level of consistency. Subjects were randomly chosen for either the control group, which used the baseline pairwise comparison method, or the experimental group, which used the alternate simultaneous comparison scale method. Each subject was measured for (1) the amount of time it took for them to read the instructions to gain a baseline understanding of the judgment task; (2) the amount of time it took for them to execute the task; (3) the internal consistency rate for their output; and (4) the accuracy of their output compared against an objective standard. The first wave of results (n = 25) found that the alternative method scored with no statistically significant difference to the baseline pairwise comparison method in terms of consistency or accuracy, and while the instructions for the alternate method took longer to read, the task execution was shorter, and thus the total time inclusive of reading and executing was not statistically significant between the two methods. These preliminary results indicate that the alternate method is a viable replacement for the baseline pairwise comparison method; however, it also yielded few advantages. Because the judgments require an objective standard to compare accuracy, a subsequent experiment could focus specifically on the consistency, varying the subjectivity and number of the criteria to determine whether a difference emerges.